Supernova11c commited on
Commit
ea975ec
·
verified ·
1 Parent(s): f2319da

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -28,4 +28,18 @@ base_model:
28
  - **Relative Speedup:** 3.52x faster than standard CPU baseline
29
  - **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
30
  - **Representation Density (StdDev):** 0.4301
31
- - supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  - **Relative Speedup:** 3.52x faster than standard CPU baseline
29
  - **Output Fidelity:** Verified torch.Size([1, 300, 2304]) with 2304-dim SANA interface.
30
  - **Representation Density (StdDev):** 0.4301
31
+ - supernova most powerful tokenizer is used here for designing the encoder making it truly power pack in a small size and convenient in CPU
32
+ ## Supernova V1 outperformed Google Gemini Embedding 001 on this evaluation suite:
33
+
34
+ 1. Retrieval Performance (Search and Match Accuracy)
35
+ Recall@1: Supernova achieved 1.6% compared to Gemini's 0.0%.
36
+ Recall@5: Supernova achieved 3.2% compared to Gemini's 0.8%.
37
+ Recall@10: Supernova achieved 5.2% compared to Gemini's 1.2%.
38
+ Mean Reciprocal Rank (MRR): Supernova reached an MRR of 0.0319, which is over 3x higher than Gemini's score of 0.0092.
39
+ 2. Semantic Similarity Alignment (Correlation with Ground Truth)
40
+ Spearman Correlation ($ ho$$ ho$): Supernova reached 0.2261, showing a much stronger monotonic relationship with human-labeled similarities than Gemini's 0.0259.
41
+ Pearson Correlation ($r$$r$): Supernova reached 0.2541, outperforming Gemini's score of 0.0366.
42
+ 3. Efficiency & Latency Trade-offs
43
+ While Google Gemini (being a lightweight cloud-hosted API) has a lower execution latency per document (~0.4ms avg vs. Supernova's local CPU latency of ~39ms avg), Supernova V1's localized architecture and official native tokenization (Supernova-NepaliFast-V4) ensure vastly superior semantic accuracy, capturing the nuances of Devanagari script, Romanized Nepali, and English code-mixed text with high precision.
44
+ ## this is a real test if you want to reproduce you can
45
+ ## data is uploaded which was used to test