File size: 6,982 Bytes
754a7c6
 
27b9539
754a7c6
 
 
 
 
ab43322
 
 
 
 
 
754a7c6
bebbd75
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
b54dd97
5a91aa5
 
 
ab43322
 
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
 
 
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
 
 
b54dd97
ab43322
b54dd97
ab43322
 
 
 
 
 
 
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
 
 
 
 
 
b54dd97
ab43322
b54dd97
 
ab43322
b54dd97
 
 
 
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
 
 
 
 
 
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
ab43322
 
b54dd97
 
ab43322
b54dd97
 
ab43322
b54dd97
 
ab43322
b54dd97
 
ab43322
 
 
b54dd97
 
ab43322
b54dd97
ab43322
b54dd97
ab43322
79959d2
 
ab43322
79959d2
 
ab43322
79959d2
 
 
ab43322
602c998
ab43322
 
 
 
602c998
 
ab43322
602c998
ab43322
 
 
 
602c998
ab43322
602c998
 
 
b54dd97
602c998
ab43322
602c998
ab43322
602c998
ab43322
 
 
 
 
 
 
 
 
 
 
 
 
 
b54dd97
ab43322
b54dd97
ab43322
 
 
 
 
 
b54dd97
ab43322
b54dd97
ab43322
b54dd97
 
754a7c6
b54dd97
 
ab43322
b54dd97
 
ab43322
b54dd97
 
ab43322
b54dd97
ab43322
 
 
 
 
b54dd97
 
 
ab43322
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
---
title: Eco-LLM Inference Routing Environment
emoji: "🌱"
colorFrom: green
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
tags:
  - openenv
  - reinforcement-learning
  - llm-routing
  - carbon-aware
  - systems
---

# Eco-LLM Inference Routing Environment

Eco-LLM Inference Routing is an OpenEnv-compatible reinforcement learning environment for carbon-aware LLM serving. The agent must decide how to route each query across model tiers and routing strategies while balancing answer quality, latency, energy use, and grid carbon intensity.

Public Space:
https://huggingface.co/spaces/rudhratej09/Eco-LLM-Carbon-Aware-LLM-Inference-Routing

after running the space for ui:
https://rudhratej09-eco-llm-carbon-aware-llm-inference-routing.hf.space/ui/

Live endpoints:
- `/tasks`: https://rudhratej09-eco-llm-carbon-aware-llm-inference-routing.hf.space/tasks
- `/metadata`: https://rudhratej09-eco-llm-carbon-aware-llm-inference-routing.hf.space/metadata
- `/docs`: https://rudhratej09-eco-llm-carbon-aware-llm-inference-routing.hf.space/docs

## Why This Environment

Modern LLM systems do not have a single objective. A router often has to decide whether a request should go to a small model, a larger model, a cache, a retrieval layer, or be delayed until carbon intensity improves. This environment turns that operational tradeoff into a deterministic RL benchmark with structured rewards and graded tasks.

The design stays close to a real serving stack:
- multiple model tiers with different energy and latency costs
- a cache that can eliminate repeat inference cost
- a knowledge-base path for FAQ-style requests
- cascade and early-exit strategies for adaptive compute
- a carbon schedule that makes timing matter

## Tasks

Three tasks are included, each with a deterministic grader returning a score in `[0.0, 1.0]`.

- `task_1`: single-query routing with a simple accuracy-versus-cost tradeoff
- `task_2`: three-query episode with an explicit penalty for overusing the large model
- `task_3`: five-query stateful episode with repeated queries, cache reuse, knowledge-base lookups, cascading, and carbon-aware waiting

Task definitions live in [server/tasks.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/tasks.py).

## Action Space

Defined in [server/models.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/models.py).

Each action has:
- `strategy`: `NONE`, `USE_CACHE`, `DO_CASCADE`, `EARLY_EXIT`, `WAIT`, or `CALL_KB`
- `model_choice`: `SMALL`, `MEDIUM`, or `LARGE`
- `exit_flag`: whether a cascade should stop early after the first correct answer

## Observation Space

Each observation includes:
- current query text
- current grid carbon intensity
- cache contents
- whether a knowledge base is available
- scalar reward
- detailed reward breakdown
- done flag
- task and state metadata

## Reward Design

The environment is dense-reward, not purely terminal.

Per step:
- `score = 1.0` when the chosen strategy reaches a model capable of solving the query, else `0.0`
- `carbon_penalty = carbon_intensity * energy_cost`, capped at `0.8`
- `latency_penalty = 0.1 * latency_cost`
- cache hits add `+0.5`
- successful early exit adds `+0.1`
- some tasks add an explicit large-model penalty

Model costs:

| Model | Energy cost | Latency cost |
|---|---:|---:|
| `SMALL` | 0.1 | 1.0 |
| `MEDIUM` | 0.3 | 2.0 |
| `LARGE` | 0.6 | 5.0 |

This makes the benchmark about policy quality, not just raw correctness.

## Core Files

- [openenv.yaml](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/openenv.yaml): OpenEnv manifest
- [server/app.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/app.py): FastAPI app and HTTP endpoints
- [server/env.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/env.py): environment logic
- [server/models.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/models.py): typed action, observation, and state models
- [server/tasks.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/tasks.py): deterministic task catalog and cost tables
- [server/rubrics.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/server/rubrics.py): rubric-style graders
- [graders.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/graders.py): top-level grader compatibility shims
- [inference.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/inference.py): root-level baseline runner
- [baseline.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/baseline.py): random, heuristic, and LLM baselines

## Local Run

Install dependencies:

```bash
uv sync
```

Start the server:

```bash
uvicorn server.app:app --host 0.0.0.0 --port 7860
```

Quick checks:

```bash
curl http://127.0.0.1:7860/health
curl http://127.0.0.1:7860/metadata
curl http://127.0.0.1:7860/tasks
```

## Baseline Inference

The required root-level baseline script is [inference.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/inference.py).

Example:

```bash
export HF_TOKEN=your_token
export API_BASE_URL=https://router.huggingface.co/v1
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
export ECO_LLM_TASK=task_3
python inference.py
```

The script emits OpenEnv-style logs:

```text
[START] task=task_3 env=eco_llm_inference_routing model=Qwen/Qwen2.5-72B-Instruct
[STEP] step=1 action=strategy=CALL_KB,model=SMALL,exit=false reward=1.30 done=false error=null
[END] success=true steps=5 score=0.72 rewards=1.30,0.48,1.50,0.20,1.90
```

## Baselines

The repository includes three reference agents:
- random policy
- heuristic router
- LLM-based router

Run comparison:

```bash
python baseline.py compare --task task_3 --episodes 20
```

These baselines are useful because they show the environment is not just a one-step classification toy. Cache-aware and carbon-aware decisions materially change returns on the harder task.

## API Surface

Main endpoints:
- `GET /`
- `GET /health`
- `GET /metadata`
- `GET /schema`
- `GET /tasks`
- `GET /tasks/meta`
- `GET /tasks/{task_id}`
- `POST /reset`
- `POST /step`
- `POST /grader`
- `POST /grade`
- `POST /grader/replay`
- `GET /docs`

## OpenEnv Compatibility

This submission includes the expected OpenEnv pieces:
- `openenv.yaml`
- root-level `inference.py`
- root-level task and grader compatibility exports
- typed action, observation, and state models
- Docker runtime for Hugging Face Spaces deployment

## Docker

Build:

```bash
docker build -t eco-llm-routing .
```

Run:

```bash
docker run -p 7860:7860 eco-llm-routing
```

## Validation Notes

The repository includes lightweight tests in [tests/test_agents.py](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/tests/test_agents.py). For a submission review, the most important checks are:
- server boots cleanly
- `/tasks`, `/metadata`, and `/docs` resolve
- the grader returns normalized scores
- the root-level inference script remains runnable

## License

BSD-style license. See [LICENSE](/C:/Users/Rudhratej/CodeMara-Meta-OpenEnv-Hackathon/LICENSE).