Update README.md
Browse files
README.md
CHANGED
|
@@ -28,4 +28,18 @@ base_model:
|
|
| 28 |
- **Relative Speedup:** 3.52x faster than standard CPU baseline
|
| 29 |
- **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
|
| 30 |
- **Representation Density (StdDev):** 0.4301
|
| 31 |
-
- supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
- **Relative Speedup:** 3.52x faster than standard CPU baseline
|
| 29 |
- **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
|
| 30 |
- **Representation Density (StdDev):** 0.4301
|
| 31 |
+
- supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
|
| 32 |
+
## Supernova V1 outperformed Google Gemini Embedding 001 on this evaluation suite:
|
| 33 |
+
|
| 34 |
+
1. Retrieval Performance (Search and Match Accuracy)
|
| 35 |
+
Recall@1: Supernova achieved 1.6% compared to Gemini's 0.0%.
|
| 36 |
+
Recall@5: Supernova achieved 3.2% compared to Gemini's 0.8%.
|
| 37 |
+
Recall@10: Supernova achieved 5.2% compared to Gemini's 1.2%.
|
| 38 |
+
Mean Reciprocal Rank (MRR): Supernova reached an MRR of 0.0319, which is over 3x higher than Gemini's score of 0.0092.
|
| 39 |
+
2. Semantic Similarity Alignment (Correlation with Ground Truth)
|
| 40 |
+
Spearman Correlation ($ ho$$ ho$): Supernova reached 0.2261, showing a much stronger monotonic relationship with human-labeled similarities than Gemini's 0.0259.
|
| 41 |
+
Pearson Correlation ($r$$r$): Supernova reached 0.2541, outperforming Gemini's score of 0.0366.
|
| 42 |
+
3. Efficiency & Latency Trade-offs
|
| 43 |
+
While Google Gemini (being a lightweight cloud-hosted API) has a lower execution latency per document (~0.4ms avg vs. Supernova's local CPU latency of ~39ms avg), Supernova V1's localized architecture and official native tokenization (Supernova-NepaliFast-V4) ensure vastly superior semantic accuracy, capturing the nuances of Devanagari script, Romanized Nepali, and English code-mixed text with high precision.
|
| 44 |
+
## this is a real test if you want to reproduce you can
|
| 45 |
+
## data is uploaded which was used to test
|