docs: remove redundant metrics and honest assessment sections
Browse files
README.md
CHANGED
|
@@ -33,7 +33,7 @@ pipeline_tag: text-generation
|
|
| 33 |
|
| 34 |
**Save Money & Time: Local Routing and Embeddings for Claude Code**
|
| 35 |
|
| 36 |
-
[Install](#-quick-start) • [Use Cases](#-what-ruvltra-is-for) • [Rust Crates](#-rust-crates) • [
|
| 37 |
|
| 38 |
```
|
| 39 |
Query → HNSW Index → GNN Layer → Enhanced Results
|
|
@@ -66,43 +66,7 @@ Those benchmarks test code generation from scratch. **RuvLTRA isn't meant to com
|
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
-
##
|
| 70 |
-
|
| 71 |
-
For routing and embeddings, you'd want to measure:
|
| 72 |
-
|
| 73 |
-
1. **Routing accuracy** - Does it pick the right agent for a task?
|
| 74 |
-
2. **Embedding quality** - Do similar tasks cluster together in HNSW?
|
| 75 |
-
3. **SONA learning rate** - Does it improve with feedback?
|
| 76 |
-
4. **Latency** - Can it make decisions in <10ms locally?
|
| 77 |
-
5. **Memory efficiency** - Can it run alongside Claude Code?
|
| 78 |
-
|
| 79 |
-
---
|
| 80 |
-
|
| 81 |
-
## 🔍 Honest Assessment
|
| 82 |
-
|
| 83 |
-
We don't know yet if RuvLTRA is better for your use case because:
|
| 84 |
-
|
| 85 |
-
- ❌ No routing accuracy benchmarks
|
| 86 |
-
- ❌ No embedding quality metrics (vs OpenAI, Cohere, etc.)
|
| 87 |
-
- ❌ No A/B testing against alternatives (e.g., `nomic-embed-text`, `bge-small`)
|
| 88 |
-
|
| 89 |
-
**What would prove it:**
|
| 90 |
-
|
| 91 |
-
```bash
|
| 92 |
-
# Create a routing benchmark
|
| 93 |
-
ruvllm benchmark routing --tasks ./test-tasks.json --expected ./expected-agents.json
|
| 94 |
-
|
| 95 |
-
# Compare embedding quality
|
| 96 |
-
ruvllm benchmark embeddings --dataset ./code-similarity-pairs.json
|
| 97 |
-
```
|
| 98 |
-
|
| 99 |
-
**What we do know:** Latency <10ms ✅ | Memory ~500MB ✅ | Cost $0 ✅
|
| 100 |
-
|
| 101 |
-
---
|
| 102 |
-
|
| 103 |
-
## 💰 Cost Savings (Why This Exists)
|
| 104 |
-
|
| 105 |
-
Every API call costs money and time:
|
| 106 |
|
| 107 |
| Operation | Claude API | RuvLTRA Local | Savings |
|
| 108 |
|-----------|------------|---------------|---------|
|
|
@@ -110,7 +74,7 @@ Every API call costs money and time:
|
|
| 110 |
| Embedding generation | $0.0001 / call | $0 | **100%** |
|
| 111 |
| Latency | ~500ms | <10ms | **50x faster** |
|
| 112 |
|
| 113 |
-
**Monthly example:** 50K routing calls = $150 saved. 100K embeddings = $10 saved. **
|
| 114 |
|
| 115 |
---
|
| 116 |
|
|
|
|
| 33 |
|
| 34 |
**Save Money & Time: Local Routing and Embeddings for Claude Code**
|
| 35 |
|
| 36 |
+
[Install](#-quick-start) • [Use Cases](#-what-ruvltra-is-for) • [Rust Crates](#-rust-crates) • [Models](#-available-models)
|
| 37 |
|
| 38 |
```
|
| 39 |
Query → HNSW Index → GNN Layer → Enhanced Results
|
|
|
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
+
## 💰 Cost Savings
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
| Operation | Claude API | RuvLTRA Local | Savings |
|
| 72 |
|-----------|------------|---------------|---------|
|
|
|
|
| 74 |
| Embedding generation | $0.0001 / call | $0 | **100%** |
|
| 75 |
| Latency | ~500ms | <10ms | **50x faster** |
|
| 76 |
|
| 77 |
+
**Monthly example:** 50K routing calls = $150 saved. 100K embeddings = $10 saved. **~$250/month.**
|
| 78 |
|
| 79 |
---
|
| 80 |
|