Buckets:

|
download
raw
10.1 kB
# REPL Environment for OpenEnv
`repl_env` is a Python REPL environment for [Recursive Language Model](https://huggingface.co/papers/2512.24601) (RLM) style execution. The model writes code that runs in a persistent namespace and can:
- inspect `context`
- execute Python across multiple turns with persistent state
- call `llm_query(...)` and `llm_query_batched(...)` to query a language model
- call `rlm_query(...)` and `rlm_query_batched(...)` for recursive child runs, when configured
- finish with `FINAL(...)`, `FINAL_VAR(...)`, or `answer = {"content": ..., "ready": True}`
The package provides:
- `REPLEnv`: the async client for a remote server (`.sync()` for synchronous code), with `execute(code)`, `submit_final_answer(answer)`, `get_variable(name)` and `list_variables()` on top of `reset()`/`step()`
- `LocalREPLEnv`: the same environment, in process
- `LocalRLMRunner`: a local RLM loop that prompts a model, runs its code and handles recursion
## Quick Start
Start a server:
```bash
PYTHONPATH=src:envs uvicorn envs.repl_env.server.app:app --host 127.0.0.1 --port 8000
```
Async:
```python
import asyncio
from repl_env import REPLEnv
async def main():
async with REPLEnv(base_url="http://127.0.0.1:8000") as env:
result = await env.reset(
context="alpha beta gamma",
task_prompt="Count the words",
)
result = await env.execute("count = len(context.split())")
result = await env.execute("print(FINAL(count))")
print(result.done)
asyncio.run(main())
```
Sync:
```python
from repl_env import REPLEnv
with REPLEnv(base_url="http://127.0.0.1:8000").sync() as env:
result = env.reset(
context="alpha beta gamma",
task_prompt="Count the words",
)
result = env.execute("count = len(context.split())")
result = env.execute("print(FINAL(count))")
print(result.observation.result.stdout)
```
### In Process
```python
from repl_env import LocalREPLEnv
with LocalREPLEnv() as env:
result = env.reset(
context="The quick brown fox jumps over the lazy dog",
task_prompt="Count the words",
)
result = env.execute("count = len(context.split())")
result = env.execute("print(FINAL(count))")
print(env.state().final_answer)
```
## Server Configuration
Environment variables read by [`server/app.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/server/app.py):
| Variable | Default | Description |
|----------|---------|-------------|
| `HF_TOKEN` | unset | Enables `llm_query` on the server. Without it, a client can pass `hf_token` to `reset()` |
| `LLM_MODEL` | `Qwen/Qwen3.5-9B` | Default model for `llm_query`. A client can pass `llm_model` to `reset()` |
| `REPL_MAX_ITERATIONS` | `30` | Maximum steps per episode |
| `REPL_MAX_OUTPUT_LENGTH` | `8192` | Maximum captured output per step |
| `REPL_CONTEXT_PREVIEW_LENGTH` | `500` | Length of `context_preview` in observations |
| `REPL_RLM_MAX_DEPTH` | `2` | Maximum recursion depth for `rlm_query` |
| `REPL_RLM_MAX_ITERATIONS` | `30` | Maximum iterations for recursive child runs |
| `MAX_CONCURRENT_ENVS` | `8` | Maximum concurrent sessions |
## Reward
Rewards use the OpenEnv [rubric system](https://huggingface.co/docs/openenv/guides/rewards). The default `REPLRubric` combines:
- **Outcome reward** (on terminal steps): compares `final_answer` against
`expected_answer` if provided. Returns 1.0 for match, 0.0 otherwise.
- **Process reward** (on non-terminal steps): returns -0.05 for code
execution errors, 0.0 for successful steps.
- **Failure reward**: returns -0.1 when max iterations exhausted without an answer.
For RL training (GRPO, etc.), pass `expected_answer` at reset time:
```python
with LocalREPLEnv() as env:
env.reset(
context="...",
task_prompt="...",
expected_answer="42", # ground truth for rubric scoring
)
result = env.execute("print(FINAL(42))")
print(result.reward) # 1.0 (correct)
```
Other rubrics: `ExactMatchRubric` (binary match), `FuzzyMatchRubric` (1.0 for an exact match, 0.5 when the expected answer is contained in the final answer), `CustomMetricRubric` (your `metric(expected, predicted) -> float`) and `CodeExecutionRubric` (per-step error penalty). Pass one at construction:
```python
from repl_env import LocalREPLEnv, CustomMetricRubric, REPLRubric
def my_metric(expected, predicted):
return 1.0 if expected.strip() == predicted.strip() else 0.0
env = LocalREPLEnv(rubric=REPLRubric(outcome=CustomMetricRubric(my_metric)))
```
## Running an RLM Locally
`LocalRLMRunner` takes any `chat_fn(messages, model=None) -> str`. It works
with HF Inference API, vLLM, SGLang, Ollama, or any OpenAI-compatible server.
With HF Inference API:
```python
from huggingface_hub import InferenceClient
from repl_env import LocalRLMRunner, RLM_SYSTEM_PROMPT
client = InferenceClient(model="Qwen/Qwen3.5-9B", timeout=300)
def chat_fn(messages, model=None):
response = client.chat.completions.create(
model=model or "Qwen/Qwen3.5-9B",
messages=messages,
max_tokens=2048,
temperature=0.6,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
return response.choices[0].message.content
runner = LocalRLMRunner(chat_fn, max_iterations=30, max_depth=2)
result = runner.run("The answer is 42", "What number is mentioned?")
print(result.final_answer)
```
With a local vLLM server:
```python
from openai import OpenAI
from repl_env import LocalRLMRunner
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
def chat_fn(messages, model=None):
response = client.chat.completions.create(
model=model or "Qwen/Qwen3.5-9B",
messages=messages,
max_tokens=2048,
temperature=0.6,
)
return response.choices[0].message.content
runner = LocalRLMRunner(chat_fn, max_iterations=30, max_depth=2)
result = runner.run(context, task)
```
### Different Models for Outer and Inner Loops
The outer loop (code generation) can use a large model while inner
`llm_query`/`rlm_query` calls use a smaller, faster model. Pass a
custom `backend_factory` to the runner:
```python
from openai import OpenAI
from huggingface_hub import InferenceClient
from repl_env import LocalRLMRunner
from repl_env.recursive_backends import BackendLimits, LocalChildRLMBackend
# Outer loop: large local model via vLLM
vllm = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
def outer_chat(messages, model=None):
r = vllm.chat.completions.create(
model="Qwen/Qwen3-32B", messages=messages, max_tokens=2048,
)
return r.choices[0].message.content
# Inner calls (llm_query/rlm_query): smaller HF-hosted model
hf = InferenceClient(model="Qwen/Qwen3.5-9B")
def inner_chat(messages, model=None):
r = hf.chat.completions.create(
model=model or "Qwen/Qwen3.5-9B", messages=messages, max_tokens=2048,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
return r.choices[0].message.content
def my_backend_factory(llm_chat_fn, **kwargs):
return LocalChildRLMBackend(
inner_chat, # inner calls use the smaller model
runner_factory=LocalRLMRunner,
system_prompt=kwargs["system_prompt"],
max_iterations=kwargs["max_iterations"],
env_max_iterations_multiplier=kwargs["env_max_iterations_multiplier"],
depth=kwargs["depth"],
limits=BackendLimits(max_depth=2),
)
runner = LocalRLMRunner(
outer_chat, # outer loop: large model
backend_factory=my_backend_factory, # inner calls: small model
max_iterations=30,
max_depth=2,
)
result = runner.run(context, task)
```
`LocalRLMRunner` also takes recursion limits (`max_children_total`, `max_children_per_batch`, `per_child_timeout_s`, `result_truncation_limit`) and lifecycle callbacks (`on_subcall_start(depth, model, prompt_preview)`, `on_subcall_complete(depth, model, duration, error_or_none)`). Its results carry lightweight child trace metadata.
## Actions and Observations
`REPLAction`
```python
code: str = ""
is_final: bool = False
final_answer: str | None = None
```
`REPLObservation`
```python
result: CodeBlockResult
context_preview: str | None
context_length: int
available_variables: list[str]
iteration: int
max_iterations: int
done: bool
reward: float | None
metadata: dict
```
## REPL Helpers
When configured, the REPL namespace exposes:
- `llm_query(prompt, model=None)` and `llm_query_batched(prompts, model=None)`
- `rlm_query(prompt, model=None)` and `rlm_query_batched(prompts, model=None)`: recursive child runs. At the maximum depth they fall back to direct model calls.
- `FINAL(value)`, `FINAL_VAR(name)` and `SHOW_VARS()`
## Finalization Patterns
### `FINAL(...)`
```python
result = env.execute("answer = 42")
result = env.execute("print(FINAL(answer))")
```
### `FINAL_VAR(...)`
```python
result = env.execute("my_answer = '42'")
result = env.execute('print(FINAL_VAR("my_answer"))')
```
### `answer` dict
```python
result = env.execute("answer['content'] = '42'")
result = env.execute("answer['ready'] = True")
```
## Prompts and Examples
[`prompts.py`](https://github.com/huggingface/OpenEnv/blob/main/envs/repl_env/prompts.py) has the system prompts and helpers used by the runner: `RLM_SYSTEM_PROMPT`, `RLM_SYSTEM_PROMPT_QWEN`, `QueryMetadata`, `build_rlm_system_prompt(...)`, `build_user_prompt(...)`, `extract_code_blocks(...)` and `format_observations(...)`.
Examples, which default to `Qwen/Qwen3.5-9B` through Hugging Face inference (needs `HF_TOKEN`):
- [`examples/repl_with_llm.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/repl_with_llm.py)
- [`examples/repl_oolong_simple.py`](https://github.com/huggingface/OpenEnv/blob/main/examples/repl_oolong_simple.py)
## References
- [RLM Paper (arXiv:2512.24601)](https://huggingface.co/papers/2512.24601)
- [RLM Implementation](https://github.com/alexzhang13/rlm)
- [Alex Zhang's RLM Blog](https://alexzhang13.github.io/blog/2025/rlm/)
- [Prime Intellect RLM Blog](https://www.primeintellect.ai/blog/rlm)

Xet Storage Details

Size:
10.1 kB
·
Xet hash:
dea28b7487034aa6d1aea34006ce5959a3d78466c2a6885aba67f6edb51b9578

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.