Update README.md
Browse files
README.md
CHANGED
|
@@ -29,18 +29,3 @@ base_model:
|
|
| 29 |
- **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
|
| 30 |
- **Representation Density (StdDev):** 0.4301
|
| 31 |
- supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
|
| 32 |
-
## Supernova V1 outperformed Google Gemini Embedding 001 on this evaluation suite:
|
| 33 |
-
|
| 34 |
-
1. Retrieval Performance (Search and Match Accuracy)
|
| 35 |
-
Recall@1: Supernova achieved 1.6% compared to Gemini's 0.0%.
|
| 36 |
-
Recall@5: Supernova achieved 3.2% compared to Gemini's 0.8%.
|
| 37 |
-
Recall@10: Supernova achieved 5.2% compared to Gemini's 1.2%.
|
| 38 |
-
Mean Reciprocal Rank (MRR): Supernova reached an MRR of 0.0319, which is over 3x higher than Gemini's score of 0.0092.
|
| 39 |
-
2. Semantic Similarity Alignment (Correlation with Ground Truth)
|
| 40 |
-
Spearman Correlation ($ ho$$ ho$): Supernova reached 0.2261, showing a much stronger monotonic relationship with human-labeled similarities than Gemini's 0.0259.
|
| 41 |
-
Pearson Correlation ($r$$r$): Supernova reached 0.2541, outperforming Gemini's score of 0.0366.
|
| 42 |
-
3. Efficiency & Latency Trade-offs
|
| 43 |
-
While Google Gemini (being a lightweight cloud-hosted API) has a lower execution latency per document (~0.4ms avg vs. Supernova's local CPU latency of ~39ms avg), Supernova V1's localized architecture and official native tokenization (Supernova-NepaliFast-V4) ensure vastly superior semantic accuracy, capturing the nuances of Devanagari script, Romanized Nepali, and English code-mixed text with high precision.
|
| 44 |
-
## this is a real test if you want to reproduce you can
|
| 45 |
-
## data is uploaded which was used to test use whole code of benchmark.py
|
| 46 |
-
## if you want to use supernova text encoder separately then you can use the code of use code.py
|
|
|
|
| 29 |
- **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
|
| 30 |
- **Representation Density (StdDev):** 0.4301
|
| 31 |
- supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|