title: FineEnvs
emoji: 🤗
colorFrom: yellow
colorTo: purple
sdk: static
pinned: false
license: mit
🤗 FineEnvs: Open RL Environments
FineEnvs is a home for end-to-end RL environment recipes, built to make it easier to explore, reproduce, train, and evaluate agent systems.
Explore complete and reproducible environment projects from us and the community, including:
- 🌍 Open RL environments
- 🧩 End-to-end environment recipes
- 💻 Complete implementations
- 📦 Models, datasets, and artifacts
- 🧪 Training and evaluation setups
- 🚀 Demos and Spaces
- 📚 Tutorials and guides
All the reproducible code — environments, rollouts, training configs, notebooks, article and slide sources — lives in one repo: github.com/adithya-s-k/FineEnvs. The artifacts those produce live here on the Hub.
FineEnvs Projects
A growing collection of open projects, environments, resources, and artifacts.
| Project | What it is | Explore |
|---|---|---|
| FineEnvs Academy | Articles, guides, tutorials, slides, and hands-on resources for learning how to build RL environments and agent systems. | Explore → |
| Data Agent | Training SLMs for data science with multi-harness RL environments. | Explore → |
| MiMo-V2.6-RL in Harbor | All 7,780 of Xiaomi's MiMo-V2.6 RL environments as Harbor tasks, plus an explorer to browse them and run graded rollouts. | Explore → · Explorer → |
| Repo2RLEnv | Verifiable coding and terminal RL environments in Harbor format, with per-task quality labels and provenance. | Explore → |
Each numbered project below is a self-contained recipe: an environment, a training run, and every artifact it produced.
| # | Project | What it is | Explore |
|---|---|---|---|
| 00 | RL Environments 101 | Three environments implemented six times over, one per framework. Same logic, six dialects. | Source → · Collection → |
| 01 | LaTeX OCR | Qwen3-VL-2B trained to read rendered math into LaTeX, scored by a reward served from a live Space. | Collection → |
| 02 | Watercolour | Qwen3.5-35B-A3B trained to paint watercolours by writing p5.brush sketches, rewarded by taste rather than correctness. | Collection → |
| 03 | GeoGuesser | A multi-turn visual geolocation environment, and the 4B trained on it until it outscored gpt-5.4-mini and claude-haiku-4.5. |
Collection → |
| 04 | SmolDataEnvs | 5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites. | Collection → |
| 05 | SmolDataEnvs: Multi-harness RL | One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints. | Article → · Slides → · Collection → |
| 06 | Multilingual | Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada. | Collection → |
| 07 | PortSimEnv | Re-plan a broken week of container-ship dockings at the Port of Barcelona, built from the port's real 2024 records and graded against a plan CP-SAT proves optimal. | Article → · Play → · Collection → |
Articles & Talks
| What it covers | Read / Watch | |
|---|---|---|
| 📖 The Ultimate Guide to RL Environments | Building and scaling RL environments in the LLM era — how frameworks are built, how rewards are wired, how they scale to thousands of concurrent sessions. | Read → |
| 🎞️ RL Environments 101 | From "what is an env?" to training your own: RL fundamentals → environment anatomy → OpenEnv → training with TRL. | Watch → |
| 📈 Scaling RL for LLMs | RL environments and RL training — what an environment is, how reward hacking happens, how to train against your own. AMD AI Dev Day. | Watch → |
| 🔀 Multi-Harness Training | OpenEnv × Harbor — why an environment's failure model decides whether it can be trained against. | Watch → |
| 🧭 The Ultimate Guide to Multi-Harness RL | Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes. | Read → |
| 🎤 Training a Coding Agent Through a Harness You Did Not Write | Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO. | Watch → |
| 🌍 How to turn a game into an RL environment | The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL. | Read → |
| 🚢 Simulation RL Environments | Turning real-world work into RL environments. Part 1 builds PortSimEnv from the Port of Barcelona's 2024 records and grades every plan against a proven optimum. | Read → |
Environments
Three reference environments, each implemented across six frameworks — openenv, ors, nemo_gym, verifiers, skyrl_gym, gem. Same logic, six dialects. Source → · RL Envs 101 collection →
| Environment | Tools | OpenEnv | ORS | NeMo Gym |
|---|---|---|---|---|
| Jupyter agent — real code execution in an E2B sandbox | 4 | Space | Space | Space |
| Wordle — multi-turn, pure Python, no backend | 1 | Space | Space | Space |
| Desktop — computer-use, vision-driven Linux desktop | 19 | Space | Space | — |
Build your own
Five agent skills turn a plain-English description into a runnable RL environment across four frameworks — works with Claude Code, Cursor, Codex, OpenCode, Gemini CLI and others.
npx skills add adithya-s-k/FineEnvs
We're looking for new end-to-end recipes — a task, an environment, a training run, and honest results. Contributing guide →
Citation
@misc{fineenvs,
author = {Kolavi, Adithya S},
title = {FineEnvs: Open Source RL Environments for LLM Agents},
year = {2026},
url = {https://github.com/adithya-s-k/FineEnvs}
}