Ypeng12 commited on
Commit
d66fa71
Β·
1 Parent(s): 5353b52

docs: rename cost efficiency paper to Hybrid LLM, add BibTeX references section, and update PDF paths to docs/article/

Browse files
README.md CHANGED
@@ -64,7 +64,7 @@ InferRoute is designed for high-throughput, low routing overhead, and self-heali
64
 
65
  InferRoute integrates the core routing methodologies and trade-off evaluation principles from the research paper:
66
  > **ROUTERBENCH: A Benchmark for Multi-LLM Routing System** (by Martian / [withmartian/routerbench](https://github.com/withmartian/routerbench)).
67
- > πŸ“„ **[Download RouterBench Paper PDF](docs/A_Benchmark_for_Multi_LLM_Routing_System.pdf)**
68
 
69
  ### 1. Mathematical Scoring & Optimization
70
  Predictive routing is formulated as choosing a backend $m$ that maximizes the utility score for a prompt $x$:
@@ -96,7 +96,7 @@ A smart, content-aware learned router (MLP/KNN) will push the Pareto frontier to
96
 
97
  InferRoute integrates the cost-performance optimization concepts from the paper:
98
  > **FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance** (by Stanford University / arXiv:2305.05196).
99
- > πŸ“„ **[Download FrugalGPT Paper PDF](docs/How_to_Use_Large_Language_Models.pdf)** | πŸ“„ **[Download Cost Efficiency PDF](docs/cost_efficiency.pdf)** | πŸ“„ **[Download InferRoute Project Plan PDF](docs/InferRoute_Project_Plan.pdf)**
100
 
101
  FrugalGPT identifies three main classes of cost-saving methods:
102
  1. **Prompt Adaptation**: Reduces input tokens dynamically. In InferRoute, when routing to cheap local backends (Ollama/vLLM), the **Prompt Adapter** (`prompt_adapter.py`) automatically compresses prompt sizes by pruning few-shot examples, restoring full prompts when upgrading to premium models.
 
64
 
65
  InferRoute integrates the core routing methodologies and trade-off evaluation principles from the research paper:
66
  > **ROUTERBENCH: A Benchmark for Multi-LLM Routing System** (by Martian / [withmartian/routerbench](https://github.com/withmartian/routerbench)).
67
+ > πŸ“„ **[Download RouterBench Paper PDF](docs/article/A_Benchmark_for_Multi_LLM_Routing_System.pdf)**
68
 
69
  ### 1. Mathematical Scoring & Optimization
70
  Predictive routing is formulated as choosing a backend $m$ that maximizes the utility score for a prompt $x$:
 
96
 
97
  InferRoute integrates the cost-performance optimization concepts from the paper:
98
  > **FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance** (by Stanford University / arXiv:2305.05196).
99
+ > πŸ“„ **[Download FrugalGPT Paper PDF](docs/article/How_to_Use_Large_Language_Models.pdf)** | πŸ“„ **[Download Hybrid LLM Routing Paper PDF](docs/article/cost_efficiency.pdf)** | πŸ“„ **[Download InferRoute Project Plan PDF](docs/InferRoute_Project_Plan.pdf)**
100
 
101
  FrugalGPT identifies three main classes of cost-saving methods:
102
  1. **Prompt Adaptation**: Reduces input tokens dynamically. In InferRoute, when routing to cheap local backends (Ollama/vLLM), the **Prompt Adapter** (`prompt_adapter.py`) automatically compresses prompt sizes by pruning few-shot examples, restoring full prompts when upgrading to premium models.
cost efficienct.pdf DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c0ebc21073112b8f8e51c7ea279eeac4d5b62be88c69a5cd2bccc2d58ec28cbb
3
- size 4833670
 
 
 
 
docs/{A_Benchmark_for_Multi_LLM_Routing_System.pdf β†’ article/A_Benchmark_for_Multi_LLM_Routing_System.pdf} RENAMED
File without changes
docs/{How_to_Use_Large_Language_Models.pdf β†’ article/How_to_Use_Large_Language_Models.pdf} RENAMED
File without changes
docs/{cost_efficiency.pdf β†’ article/cost_efficiency.pdf} RENAMED
File without changes
docs/index.html CHANGED
@@ -785,22 +785,22 @@
785
  <h2>πŸ“š Original Research Papers & Download Hub</h2>
786
  <p>Read or download the full research publications and project blueprints associated with this routing engine:</p>
787
  <div class="paper-grid" style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 1rem; margin-top: 1.5rem;">
788
- <a href="A_Benchmark_for_Multi_LLM_Routing_System.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
789
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
790
  <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">RouterBench Paper</span>
791
  <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">A Benchmark for Multi-LLM Routing System (Martian, 2024)</span>
792
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
793
  </a>
794
- <a href="How_to_Use_Large_Language_Models.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
795
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
796
  <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">FrugalGPT Paper</span>
797
  <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">How to Use Large Language Models While Reducing Cost and Improving Performance (Stanford, 2023)</span>
798
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
799
  </a>
800
- <a href="cost_efficiency.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
801
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
802
- <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">Cost Efficiency Paper</span>
803
- <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">Cost-efficient execution strategies of Large Language Models.</span>
804
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
805
  </a>
806
  <a href="InferRoute_Project_Plan.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
@@ -811,6 +811,64 @@
811
  </a>
812
  </div>
813
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
814
  </div>
815
 
816
  <!-- Tab 2: Architecture -->
 
785
  <h2>πŸ“š Original Research Papers & Download Hub</h2>
786
  <p>Read or download the full research publications and project blueprints associated with this routing engine:</p>
787
  <div class="paper-grid" style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 1rem; margin-top: 1.5rem;">
788
+ <a href="article/A_Benchmark_for_Multi_LLM_Routing_System.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
789
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
790
  <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">RouterBench Paper</span>
791
  <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">A Benchmark for Multi-LLM Routing System (Martian, 2024)</span>
792
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
793
  </a>
794
+ <a href="article/How_to_Use_Large_Language_Models.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
795
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
796
  <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">FrugalGPT Paper</span>
797
  <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">How to Use Large Language Models While Reducing Cost and Improving Performance (Stanford, 2023)</span>
798
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
799
  </a>
800
+ <a href="article/cost_efficiency.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
801
  <span style="font-size: 1.5rem; margin-bottom: 0.5rem;">πŸ“„</span>
802
+ <span style="font-family: var(--font-outfit); font-weight: 700; font-size: 1.1rem; color: var(--text-primary); margin-bottom: 0.25rem;">Hybrid LLM Routing Paper</span>
803
+ <span style="font-size: 0.8rem; color: var(--text-secondary); margin-bottom: 0.75rem;">Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing (IBM / Tsinghua, 2024)</span>
804
  <span style="align-self: flex-start; background: rgba(139, 92, 246, 0.15); border: 1px solid rgba(109, 40, 217, 0.2); color: var(--accent-violet); font-size: 0.75rem; font-weight: 600; padding: 0.25rem 0.65rem; border-radius: 6px;">Download PDF</span>
805
  </a>
806
  <a href="InferRoute_Project_Plan.pdf" download class="sub-card" style="display: flex; flex-direction: column; text-decoration: none; transition: all 0.3s; padding: 1.5rem; background: rgba(0, 0, 0, 0.02); border: 1px solid var(--border-color); border-radius: 12px; color: var(--text-primary);">
 
811
  </a>
812
  </div>
813
  </div>
814
+
815
+ <!-- Academic Bibliography & References -->
816
+ <div class="glass-card" style="margin-top: 2rem;">
817
+ <h2>πŸ“š Academic Bibliography & References</h2>
818
+ <p>Formal scientific citations for the core research papers referenced during the design and optimization of the InferRoute gateway:</p>
819
+
820
+ <div style="display: flex; flex-direction: column; gap: 1.5rem; margin-top: 1.5rem;">
821
+
822
+ <div class="sub-card" style="background: rgba(0,0,0,0.01);">
823
+ <h4 style="font-family: var(--font-outfit); font-weight: 700; color: var(--text-primary); margin-bottom: 0.5rem;">1. FrugalGPT (Stanford University)</h4>
824
+ <p style="font-size: 0.9rem; color: var(--text-secondary); margin-bottom: 0.75rem; font-style: italic;">
825
+ Chen, L., Zaharia, M., & Zou, J. (2023). FrugalGPT: How to use large language models while reducing cost and improving performance. <em>arXiv preprint arXiv:2305.05196</em>.
826
+ </p>
827
+ <details style="cursor: pointer; font-size: 0.85rem;">
828
+ <summary style="color: var(--accent-violet); font-weight: 600; outline: none; margin-bottom: 0.5rem;">Show BibTeX Citation</summary>
829
+ <pre style="background: rgba(0,0,0,0.03); padding: 0.75rem; border-radius: 6px; border: 1px solid var(--border-color); overflow-x: auto; font-family: monospace; color: var(--text-secondary);">@article{chen2023frugalgpt,
830
+ title={FrugalGPT: How to use large language models while reducing cost and improving performance},
831
+ author={Chen, Lingjiao and Zaharia, Matei and Zou, James},
832
+ journal={arXiv preprint arXiv:2305.05196},
833
+ year={2023}
834
+ }</pre>
835
+ </details>
836
+ </div>
837
+
838
+ <div class="sub-card" style="background: rgba(0,0,0,0.01);">
839
+ <h4 style="font-family: var(--font-outfit); font-weight: 700; color: var(--text-primary); margin-bottom: 0.5rem;">2. RouterBench (Martian)</h4>
840
+ <p style="font-size: 0.9rem; color: var(--text-secondary); margin-bottom: 0.75rem; font-style: italic;">
841
+ Li, T., Martian Team, et al. (2024). RouterBench: A Benchmark for Multi-LLM Routing System. <em>arXiv preprint arXiv:2403.11164</em>.
842
+ </p>
843
+ <details style="cursor: pointer; font-size: 0.85rem;">
844
+ <summary style="color: var(--accent-violet); font-weight: 600; outline: none; margin-bottom: 0.5rem;">Show BibTeX Citation</summary>
845
+ <pre style="background: rgba(0,0,0,0.03); padding: 0.75rem; border-radius: 6px; border: 1px solid var(--border-color); overflow-x: auto; font-family: monospace; color: var(--text-secondary);">@article{li2024routerbench,
846
+ title={RouterBench: A Benchmark for Multi-LLM Routing System},
847
+ author={Li, Teh-Hsien and others},
848
+ journal={arXiv preprint arXiv:2403.11164},
849
+ year={2024}
850
+ }</pre>
851
+ </details>
852
+ </div>
853
+
854
+ <div class="sub-card" style="background: rgba(0,0,0,0.01);">
855
+ <h4 style="font-family: var(--font-outfit); font-weight: 700; color: var(--text-primary); margin-bottom: 0.5rem;">3. Hybrid LLM Routing (IBM / Tsinghua)</h4>
856
+ <p style="font-size: 0.9rem; color: var(--text-secondary); margin-bottom: 0.75rem; font-style: italic;">
857
+ Ding, J., et al. (2024). Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing. <em>arXiv preprint arXiv:2404.14944</em>.
858
+ </p>
859
+ <details style="cursor: pointer; font-size: 0.85rem;">
860
+ <summary style="color: var(--accent-violet); font-weight: 600; outline: none; margin-bottom: 0.5rem;">Show BibTeX Citation</summary>
861
+ <pre style="background: rgba(0,0,0,0.03); padding: 0.75rem; border-radius: 6px; border: 1px solid var(--border-color); overflow-x: auto; font-family: monospace; color: var(--text-secondary);">@article{ding2024hybrid,
862
+ title={Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing},
863
+ author={Ding, Jiayi and others},
864
+ journal={arXiv preprint arXiv:2404.14944},
865
+ year={2024}
866
+ }</pre>
867
+ </details>
868
+ </div>
869
+
870
+ </div>
871
+ </div>
872
  </div>
873
 
874
  <!-- Tab 2: Architecture -->