Upload fine-tuned chart reranker model
Browse files- README.md +67 -68
- eval/CrossEncoderCorrelationEvaluator_validation_results.csv +5 -5
- model.safetensors +1 -1
- training_info.txt +1 -1
README.md
CHANGED
|
@@ -4,7 +4,7 @@ tags:
|
|
| 4 |
- cross-encoder
|
| 5 |
- reranker
|
| 6 |
- generated_from_trainer
|
| 7 |
-
- dataset_size:
|
| 8 |
- loss:BinaryCrossEntropyLoss
|
| 9 |
base_model: Alibaba-NLP/gte-multilingual-reranker-base
|
| 10 |
pipeline_tag: text-ranking
|
|
@@ -23,10 +23,10 @@ model-index:
|
|
| 23 |
type: validation
|
| 24 |
metrics:
|
| 25 |
- type: pearson
|
| 26 |
-
value: 0.
|
| 27 |
name: Pearson
|
| 28 |
- type: spearman
|
| 29 |
-
value: 0.
|
| 30 |
name: Spearman
|
| 31 |
---
|
| 32 |
|
|
@@ -70,11 +70,11 @@ from sentence_transformers import CrossEncoder
|
|
| 70 |
model = CrossEncoder("cross_encoder_model_id")
|
| 71 |
# Get scores for pairs of texts
|
| 72 |
pairs = [
|
| 73 |
-
[
|
| 74 |
-
['
|
| 75 |
-
['
|
| 76 |
-
['
|
| 77 |
-
['
|
| 78 |
]
|
| 79 |
scores = model.predict(pairs)
|
| 80 |
print(scores.shape)
|
|
@@ -82,13 +82,13 @@ print(scores.shape)
|
|
| 82 |
|
| 83 |
# Or rank different texts based on similarity to a single text
|
| 84 |
ranks = model.rank(
|
| 85 |
-
|
| 86 |
[
|
| 87 |
-
'Title: "
|
| 88 |
-
'Title: "
|
| 89 |
-
'Title: "
|
| 90 |
-
'Title: "
|
| 91 |
-
'Title: "
|
| 92 |
]
|
| 93 |
)
|
| 94 |
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
|
|
@@ -129,8 +129,8 @@ You can finetune this model on your own dataset.
|
|
| 129 |
|
| 130 |
| Metric | Value |
|
| 131 |
|:-------------|:-----------|
|
| 132 |
-
| pearson | 0.
|
| 133 |
-
| **spearman** | **0.
|
| 134 |
|
| 135 |
<!--
|
| 136 |
## Bias, Risks and Limitations
|
|
@@ -150,19 +150,19 @@ You can finetune this model on your own dataset.
|
|
| 150 |
|
| 151 |
#### Unnamed Dataset
|
| 152 |
|
| 153 |
-
* Size:
|
| 154 |
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
|
| 155 |
* Approximate statistics based on the first 1000 samples:
|
| 156 |
-
| | sentence_0
|
| 157 |
-
|:--------|:----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
|
| 158 |
-
| type | string
|
| 159 |
-
| details | <ul><li>min:
|
| 160 |
* Samples:
|
| 161 |
-
| sentence_0
|
| 162 |
-
|:--------------------------------------------------------------------------------------------------------------
|
| 163 |
-
| <code>
|
| 164 |
-
| <code>
|
| 165 |
-
| <code>
|
| 166 |
* Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
|
| 167 |
```json
|
| 168 |
{
|
|
@@ -308,48 +308,47 @@ You can finetune this model on your own dataset.
|
|
| 308 |
### Training Logs
|
| 309 |
| Epoch | Step | Training Loss | validation_spearman |
|
| 310 |
|:------:|:----:|:-------------:|:-------------------:|
|
| 311 |
-
| 0.
|
| 312 |
-
| 0.
|
| 313 |
-
| 0.
|
| 314 |
-
| 0.
|
| 315 |
-
| 0.
|
| 316 |
-
| 0.
|
| 317 |
-
| 0.
|
| 318 |
-
| 0.
|
| 319 |
-
| 1.0 |
|
| 320 |
-
| 1.
|
| 321 |
-
| 1.
|
| 322 |
-
| 1.
|
| 323 |
-
| 1.
|
| 324 |
-
| 1.
|
| 325 |
-
| 1.
|
| 326 |
-
| 1.
|
| 327 |
-
| 1.
|
| 328 |
-
| 1.
|
| 329 |
-
| 2.0 |
|
| 330 |
-
| 2.
|
| 331 |
-
| 2.
|
| 332 |
-
| 2.
|
| 333 |
-
| 2.
|
| 334 |
-
| 2.
|
| 335 |
-
| 2.
|
| 336 |
-
| 2.
|
| 337 |
-
| 2.
|
| 338 |
-
| 2.
|
| 339 |
-
| 3.0 |
|
| 340 |
-
| 3.
|
| 341 |
-
| 3.
|
| 342 |
-
| 3.
|
| 343 |
-
| 3.
|
| 344 |
-
| 3.
|
| 345 |
-
| 3.
|
| 346 |
-
| 3.
|
| 347 |
-
| 3.
|
| 348 |
-
|
|
| 349 |
-
| 4.
|
| 350 |
-
| 4.
|
| 351 |
-
| 4.
|
| 352 |
-
| 4.4571 | 3900 | - | 0.8672 |
|
| 353 |
|
| 354 |
|
| 355 |
### Framework Versions
|
|
|
|
| 4 |
- cross-encoder
|
| 5 |
- reranker
|
| 6 |
- generated_from_trainer
|
| 7 |
+
- dataset_size:28274
|
| 8 |
- loss:BinaryCrossEntropyLoss
|
| 9 |
base_model: Alibaba-NLP/gte-multilingual-reranker-base
|
| 10 |
pipeline_tag: text-ranking
|
|
|
|
| 23 |
type: validation
|
| 24 |
metrics:
|
| 25 |
- type: pearson
|
| 26 |
+
value: 0.871938379575355
|
| 27 |
name: Pearson
|
| 28 |
- type: spearman
|
| 29 |
+
value: 0.8696556409896702
|
| 30 |
name: Spearman
|
| 31 |
---
|
| 32 |
|
|
|
|
| 70 |
model = CrossEncoder("cross_encoder_model_id")
|
| 71 |
# Get scores for pairs of texts
|
| 72 |
pairs = [
|
| 73 |
+
['Which securities listed on ENXTAM were the top performers (highest returns) in the period shown?', 'Title: "Pedro\'s List Financials"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Financials"="Financials Overview", "Pedro\'s List"="Pedro\'s List, Inc."\nSources: S&P Global'],
|
| 74 |
+
['Best crypto platforms compliant with MiCA 2025, reliable, secure, and transparent.', 'Title: "Top Performing Crypto"\nCollections: Crypto Currencies\nDatasets: CryptoAssetMetrics\nChart Type: categorical_bar'],
|
| 75 |
+
['implications fiscales et sociales formes juridiques entreprises France 2025 apports industrie', 'Title: "Eagle Industries Overview"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Eagle Industries"="Eagle Industry Co.,Ltd.", "Overview"="Stock Overview"\nSources: S&P Global'],
|
| 76 |
+
['Which US-listed stocks on the NYSE and NASDAQ are the top performers (highest returns)?', 'Title: "Nasdaq Inc. Financials"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Nasdaq Inc."="Nasdaq, Inc.", "Financials"="Financials Overview"\nSources: S&P Global'],
|
| 77 |
+
['How many children are recorded with wheezing, and has that number gone up or down over time?', 'Title: "Mani Number (Annual), Universal Music Number (Quarterly)"\nCollections: Companies\nDatasets: StandardIncomeStatement\nChart Type: timeseries:eav_v3\nCanonical forms: "Number"="inventory"'],
|
| 78 |
]
|
| 79 |
scores = model.predict(pairs)
|
| 80 |
print(scores.shape)
|
|
|
|
| 82 |
|
| 83 |
# Or rank different texts based on similarity to a single text
|
| 84 |
ranks = model.rank(
|
| 85 |
+
'Which securities listed on ENXTAM were the top performers (highest returns) in the period shown?',
|
| 86 |
[
|
| 87 |
+
'Title: "Pedro\'s List Financials"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Financials"="Financials Overview", "Pedro\'s List"="Pedro\'s List, Inc."\nSources: S&P Global',
|
| 88 |
+
'Title: "Top Performing Crypto"\nCollections: Crypto Currencies\nDatasets: CryptoAssetMetrics\nChart Type: categorical_bar',
|
| 89 |
+
'Title: "Eagle Industries Overview"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Eagle Industries"="Eagle Industry Co.,Ltd.", "Overview"="Stock Overview"\nSources: S&P Global',
|
| 90 |
+
'Title: "Nasdaq Inc. Financials"\nCollections: Companies\nChart Type: company:finance\nCanonical forms: "Nasdaq Inc."="Nasdaq, Inc.", "Financials"="Financials Overview"\nSources: S&P Global',
|
| 91 |
+
'Title: "Mani Number (Annual), Universal Music Number (Quarterly)"\nCollections: Companies\nDatasets: StandardIncomeStatement\nChart Type: timeseries:eav_v3\nCanonical forms: "Number"="inventory"',
|
| 92 |
]
|
| 93 |
)
|
| 94 |
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
|
|
|
|
| 129 |
|
| 130 |
| Metric | Value |
|
| 131 |
|:-------------|:-----------|
|
| 132 |
+
| pearson | 0.8719 |
|
| 133 |
+
| **spearman** | **0.8697** |
|
| 134 |
|
| 135 |
<!--
|
| 136 |
## Bias, Risks and Limitations
|
|
|
|
| 150 |
|
| 151 |
#### Unnamed Dataset
|
| 152 |
|
| 153 |
+
* Size: 28,274 training samples
|
| 154 |
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
|
| 155 |
* Approximate statistics based on the first 1000 samples:
|
| 156 |
+
| | sentence_0 | sentence_1 | label |
|
| 157 |
+
|:--------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
|
| 158 |
+
| type | string | string | float |
|
| 159 |
+
| details | <ul><li>min: 3 characters</li><li>mean: 82.52 characters</li><li>max: 939 characters</li></ul> | <ul><li>min: 75 characters</li><li>mean: 171.07 characters</li><li>max: 436 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.43</li><li>max: 1.0</li></ul> |
|
| 160 |
* Samples:
|
| 161 |
+
| sentence_0 | sentence_1 | label |
|
| 162 |
+
|:--------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------|
|
| 163 |
+
| <code>Which securities listed on ENXTAM were the top performers (highest returns) in the period shown?</code> | <code>Title: "Pedro's List Financials"<br>Collections: Companies<br>Chart Type: company:finance<br>Canonical forms: "Financials"="Financials Overview", "Pedro's List"="Pedro's List, Inc."<br>Sources: S&P Global</code> | <code>0.0</code> |
|
| 164 |
+
| <code>Best crypto platforms compliant with MiCA 2025, reliable, secure, and transparent.</code> | <code>Title: "Top Performing Crypto"<br>Collections: Crypto Currencies<br>Datasets: CryptoAssetMetrics<br>Chart Type: categorical_bar</code> | <code>0.25</code> |
|
| 165 |
+
| <code>implications fiscales et sociales formes juridiques entreprises France 2025 apports industrie</code> | <code>Title: "Eagle Industries Overview"<br>Collections: Companies<br>Chart Type: company:finance<br>Canonical forms: "Eagle Industries"="Eagle Industry Co.,Ltd.", "Overview"="Stock Overview"<br>Sources: S&P Global</code> | <code>0.0</code> |
|
| 166 |
* Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
|
| 167 |
```json
|
| 168 |
{
|
|
|
|
| 308 |
### Training Logs
|
| 309 |
| Epoch | Step | Training Loss | validation_spearman |
|
| 310 |
|:------:|:----:|:-------------:|:-------------------:|
|
| 311 |
+
| 0.1131 | 100 | - | 0.7659 |
|
| 312 |
+
| 0.2262 | 200 | - | 0.7941 |
|
| 313 |
+
| 0.3394 | 300 | - | 0.8119 |
|
| 314 |
+
| 0.4525 | 400 | - | 0.8237 |
|
| 315 |
+
| 0.5656 | 500 | 0.4773 | 0.8284 |
|
| 316 |
+
| 0.6787 | 600 | - | 0.8304 |
|
| 317 |
+
| 0.7919 | 700 | - | 0.8361 |
|
| 318 |
+
| 0.9050 | 800 | - | 0.8454 |
|
| 319 |
+
| 1.0 | 884 | - | 0.8382 |
|
| 320 |
+
| 1.0181 | 900 | - | 0.8438 |
|
| 321 |
+
| 1.1312 | 1000 | 0.4184 | 0.8469 |
|
| 322 |
+
| 1.2443 | 1100 | - | 0.8458 |
|
| 323 |
+
| 1.3575 | 1200 | - | 0.8492 |
|
| 324 |
+
| 1.4706 | 1300 | - | 0.8514 |
|
| 325 |
+
| 1.5837 | 1400 | - | 0.8567 |
|
| 326 |
+
| 1.6968 | 1500 | 0.3897 | 0.8582 |
|
| 327 |
+
| 1.8100 | 1600 | - | 0.8586 |
|
| 328 |
+
| 1.9231 | 1700 | - | 0.8582 |
|
| 329 |
+
| 2.0 | 1768 | - | 0.8587 |
|
| 330 |
+
| 2.0362 | 1800 | - | 0.8583 |
|
| 331 |
+
| 2.1493 | 1900 | - | 0.8597 |
|
| 332 |
+
| 2.2624 | 2000 | 0.3709 | 0.8596 |
|
| 333 |
+
| 2.3756 | 2100 | - | 0.8608 |
|
| 334 |
+
| 2.4887 | 2200 | - | 0.8598 |
|
| 335 |
+
| 2.6018 | 2300 | - | 0.8623 |
|
| 336 |
+
| 2.7149 | 2400 | - | 0.8643 |
|
| 337 |
+
| 2.8281 | 2500 | 0.3556 | 0.8661 |
|
| 338 |
+
| 2.9412 | 2600 | - | 0.8672 |
|
| 339 |
+
| 3.0 | 2652 | - | 0.8656 |
|
| 340 |
+
| 3.0543 | 2700 | - | 0.8668 |
|
| 341 |
+
| 3.1674 | 2800 | - | 0.8657 |
|
| 342 |
+
| 3.2805 | 2900 | - | 0.8654 |
|
| 343 |
+
| 3.3937 | 3000 | 0.3435 | 0.8656 |
|
| 344 |
+
| 3.5068 | 3100 | - | 0.8665 |
|
| 345 |
+
| 3.6199 | 3200 | - | 0.8661 |
|
| 346 |
+
| 3.7330 | 3300 | - | 0.8660 |
|
| 347 |
+
| 3.8462 | 3400 | - | 0.8666 |
|
| 348 |
+
| 3.9593 | 3500 | 0.3364 | 0.8679 |
|
| 349 |
+
| 4.0 | 3536 | - | 0.8674 |
|
| 350 |
+
| 4.0724 | 3600 | - | 0.8670 |
|
| 351 |
+
| 4.1855 | 3700 | - | 0.8697 |
|
|
|
|
| 352 |
|
| 353 |
|
| 354 |
### Framework Versions
|
eval/CrossEncoderCorrelationEvaluator_validation_results.csv
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
epoch,steps,Pearson_Correlation,Spearman_Correlation
|
| 2 |
-
1.0,
|
| 3 |
-
2.0,
|
| 4 |
-
3.0,
|
| 5 |
-
4.0,
|
| 6 |
-
5.0,
|
|
|
|
| 1 |
epoch,steps,Pearson_Correlation,Spearman_Correlation
|
| 2 |
+
1.0,884,0.8401566336608757,0.8382300998214652
|
| 3 |
+
2.0,1768,0.8615358688421247,0.8587285555844401
|
| 4 |
+
3.0,2652,0.8681246217823901,0.8655791469764533
|
| 5 |
+
4.0,3536,0.8692562599529016,0.8674157368971115
|
| 6 |
+
5.0,4420,0.8707395217729429,0.8684691714077699
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1223854204
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:922f11cc3ffaaa5abc058dadce6044e126ff6b5b5c6595408217073cd2d86548
|
| 3 |
size 1223854204
|
training_info.txt
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
Base Model: Alibaba-NLP/gte-multilingual-reranker-base
|
| 2 |
-
Training Samples:
|
| 3 |
Epochs: 5
|
| 4 |
Batch Size: 32
|
| 5 |
Learning Rate: 2e-05
|
|
|
|
| 1 |
Base Model: Alibaba-NLP/gte-multilingual-reranker-base
|
| 2 |
+
Training Samples: 28274
|
| 3 |
Epochs: 5
|
| 4 |
Batch Size: 32
|
| 5 |
Learning Rate: 2e-05
|