Supernova11c commited on
Commit
82fc14d
·
verified ·
1 Parent(s): f1e821c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -15
README.md CHANGED
@@ -29,18 +29,3 @@ base_model:
29
  - **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
30
  - **Representation Density (StdDev):** 0.4301
31
  - supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
32
- ## Supernova V1 outperformed Google Gemini Embedding 001 on this evaluation suite:
33
-
34
- 1. Retrieval Performance (Search and Match Accuracy)
35
- Recall@1: Supernova achieved 1.6% compared to Gemini's 0.0%.
36
- Recall@5: Supernova achieved 3.2% compared to Gemini's 0.8%.
37
- Recall@10: Supernova achieved 5.2% compared to Gemini's 1.2%.
38
- Mean Reciprocal Rank (MRR): Supernova reached an MRR of 0.0319, which is over 3x higher than Gemini's score of 0.0092.
39
- 2. Semantic Similarity Alignment (Correlation with Ground Truth)
40
- Spearman Correlation ($ ho$$ ho$): Supernova reached 0.2261, showing a much stronger monotonic relationship with human-labeled similarities than Gemini's 0.0259.
41
- Pearson Correlation ($r$$r$): Supernova reached 0.2541, outperforming Gemini's score of 0.0366.
42
- 3. Efficiency & Latency Trade-offs
43
- While Google Gemini (being a lightweight cloud-hosted API) has a lower execution latency per document (~0.4ms avg vs. Supernova's local CPU latency of ~39ms avg), Supernova V1's localized architecture and official native tokenization (Supernova-NepaliFast-V4) ensure vastly superior semantic accuracy, capturing the nuances of Devanagari script, Romanized Nepali, and English code-mixed text with high precision.
44
- ## this is a real test if you want to reproduce you can
45
- ## data is uploaded which was used to test use whole code of benchmark.py
46
- ## if you want to use supernova text encoder separately then you can use the code of use code.py
 
29
  - **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
30
  - **Representation Density (StdDev):** 0.4301
31
  - supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU