File size: 10,140 Bytes
444d4b3 b41df73 93a49ec 444d4b3 93a49ec 444d4b3 6419981 93a49ec b41df73 4ba9390 93a49ec b41df73 93a49ec b41df73 93a49ec b41df73 4ba9390 93a49ec b41df73 93a49ec 9f79950 93a49ec 584fc5c 9f79950 b41df73 9f79950 c1e9830 21935d6 e4d28dd 584fc5c 4ba9390 9f79950 b41df73 e4d28dd 4ba9390 9f79950 4ba9390 9f79950 4ba9390 b41df73 4ba9390 b41df73 2c63aa3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | ---
title: FineEnvs
emoji: 🤗
colorFrom: yellow
colorTo: purple
sdk: static
pinned: false
license: mit
---
[](https://github.com/adithya-s-k/FineEnvs)
[](https://huggingface.co/spaces/AdithyaSK/rl-environments-guide)
[](https://huggingface.co/spaces/AdithyaSK/rl-environments-101-slides)
# 🤗 FineEnvs: Open RL Environments
FineEnvs is a home for **end-to-end RL environment recipes**, built to make it easier to **explore, reproduce, train, and evaluate agent systems**.
Explore complete and reproducible environment projects from us and the community, including:
* 🌍 **Open RL environments**
* 🧩 **End-to-end environment recipes**
* 💻 **Complete implementations**
* 📦 **Models, datasets, and artifacts**
* 🧪 **Training and evaluation setups**
* 🚀 **Demos and Spaces**
* 📚 **Tutorials and guides**
All the reproducible code — environments, rollouts, training configs, notebooks, article and slide sources — lives in one repo: **[github.com/adithya-s-k/FineEnvs](https://github.com/adithya-s-k/FineEnvs)**. The artifacts those produce live here on the Hub.
# FineEnvs Projects
A growing collection of open projects, environments, resources, and artifacts.
| Project | What it is | Explore |
| :------------------------- | :----------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------ |
| **FineEnvs Academy** | Articles, guides, tutorials, slides, and hands-on resources for learning how to build RL environments and agent systems. | [Explore →](https://huggingface.co/collections/FineEnvs/fineenvs-academy-6a942aed464a2eaa85047596) |
| **Data Agent** | Training SLMs for data science with multi-harness RL environments. | [Explore →](https://huggingface.co/collections/FineEnvs/data-agent-6a942bca75a18973f8285023) |
| **MiMo-V2.6-RL in Harbor** | All 7,780 of Xiaomi's MiMo-V2.6 RL environments as Harbor tasks, plus an explorer to browse them and run graded rollouts. | [Explore →](https://huggingface.co/collections/FineEnvs/mimo-v26-rl-in-harbor-6ab836dba014bd7580f8a3fe) · [Explorer →](https://huggingface.co/spaces/FineEnvs/MiMo-RL-Envs-Explorer) |
| **Repo2RLEnv** | Verifiable coding and terminal RL environments in Harbor format, with per-task quality labels and provenance. | [Explore →](https://huggingface.co/collections/FineEnvs/repo2rlenv-verifiable-rl-environments-6aa82300d7494c050f50508d) |
Each numbered project below is a self-contained recipe: an environment, a training run, and every artifact it produced.
| # | Project | What it is | Explore |
| :-- | :--- | :--- | :--- |
| **00** | **RL Environments 101** | Three environments implemented six times over, one per framework. Same logic, six dialects. | [Source →](https://github.com/adithya-s-k/FineEnvs/tree/main/00-environments-101) · [Collection →](https://huggingface.co/collections/FineEnvs/rl-envs-101-6abca516b22e765e0b1aa0d1) |
| **01** | **LaTeX OCR** | Qwen3-VL-2B trained to read rendered math into LaTeX, scored by a reward served from a live Space. | [Collection →](https://huggingface.co/collections/FineEnvs/latex-ocr-6aa7ed8498b3ffd222ad1c8c) |
| **02** | **Watercolour** | Qwen3.5-35B-A3B trained to paint watercolours by writing p5.brush sketches, rewarded by taste rather than correctness. | [Collection →](https://huggingface.co/collections/FineEnvs/paint-with-code-6a955b79d63f67f1631d9be6) |
| **03** | **GeoGuesser** | A multi-turn visual geolocation environment, and the 4B trained on it until it outscored `gpt-5.4-mini` and `claude-haiku-4.5`. | [Collection →](https://huggingface.co/collections/FineEnvs/geoguesser-env-6a969f8db267fe0e85fa1ab6) |
| **04** | **SmolDataEnvs** | 5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites. | [Collection →](https://huggingface.co/collections/FineEnvs/smoldataenvs-6ab4f2f6e09b7cb872ebc867) |
| **05** | **SmolDataEnvs: Multi-harness RL** | One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints. | [Article →](https://huggingface.co/spaces/FineEnvs/multi-harness-rl) · [Slides →](https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) · [Collection →](https://huggingface.co/collections/FineEnvs/smoldataenvs-multi-harness-rl-6abdfaaa8d74dacd481d5212) |
| **06** | **Multilingual** | Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada. | [Collection →](https://huggingface.co/collections/FineEnvs/multilingual-multimodal-envs-6ac0c27c137f93e0799603e4) |
| **07** | **PortSimEnv** | Re-plan a broken week of container-ship dockings at the Port of Barcelona, built from the port's real 2024 records and graded against a plan CP-SAT proves optimal. | [Article →](https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) · [Play →](https://huggingface.co/spaces/FineEnvs/PortSimEnv) · [Collection →](https://huggingface.co/collections/FineEnvs/simulation-rl-envs) |
# Articles & Talks
| | What it covers | Read / Watch |
| :--- | :--- | :--- |
| 📖 **The Ultimate Guide to RL Environments** | Building and scaling RL environments in the LLM era — how frameworks are built, how rewards are wired, how they scale to thousands of concurrent sessions. | [Read →](https://huggingface.co/spaces/AdithyaSK/rl-environments-guide) |
| 🎞️ **RL Environments 101** | From "what is an env?" to training your own: RL fundamentals → environment anatomy → OpenEnv → training with TRL. | [Watch →](https://huggingface.co/spaces/AdithyaSK/rl-environments-101-slides) |
| 📈 **Scaling RL for LLMs** | RL environments and RL training — what an environment is, how reward hacking happens, how to train against your own. AMD AI Dev Day. | [Watch →](https://huggingface.co/spaces/AdithyaSK/scaling-rl-for-llms-amd-ai-dev-day) |
| 🔀 **Multi-Harness Training** | OpenEnv × Harbor — why an environment's failure model decides whether it can be trained against. | [Watch →](https://huggingface.co/spaces/AdithyaSK/multi-harness-training-slides) |
| 🧭 **The Ultimate Guide to Multi-Harness RL** | Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes. | [Read →](https://huggingface.co/spaces/FineEnvs/multi-harness-rl) |
| 🎤 **Training a Coding Agent Through a Harness You Did Not Write** | Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO. | [Watch →](https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) |
| 🌍 **How to turn a game into an RL environment** | The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL. | [Read →](https://huggingface.co/spaces/FineEnvs/geoguesser-article) |
| 🚢 **Simulation RL Environments** | Turning real-world work into RL environments. Part 1 builds PortSimEnv from the Port of Barcelona's 2024 records and grades every plan against a proven optimum. | [Read →](https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) |
# Environments
Three reference environments, each implemented across six frameworks — `openenv`, `ors`, `nemo_gym`, `verifiers`, `skyrl_gym`, `gem`. Same logic, six dialects. [Source →](https://github.com/adithya-s-k/FineEnvs/tree/main/00-environments-101) · [RL Envs 101 collection →](https://huggingface.co/collections/FineEnvs/rl-envs-101-6abca516b22e765e0b1aa0d1)
| Environment | Tools | OpenEnv | ORS | NeMo Gym |
| :--- | :--: | :--- | :--- | :--- |
| **Jupyter agent** — real code execution in an E2B sandbox | 4 | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-openenv) | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-ors) | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-nemo-gym) |
| **Wordle** — multi-turn, pure Python, no backend | 1 | [Space](https://huggingface.co/spaces/FineEnvs/wordle-openenv) | [Space](https://huggingface.co/spaces/FineEnvs/wordle-ors) | [Space](https://huggingface.co/spaces/FineEnvs/wordle-nemo-gym) |
| **Desktop** — computer-use, vision-driven Linux desktop | 19 | [Space](https://huggingface.co/spaces/AdithyaSK/desktop-openenv) | [Space](https://huggingface.co/spaces/AdithyaSK/desktop-ors) | — |
# Build your own
Five agent skills turn a plain-English description into a runnable RL environment across four frameworks — works with Claude Code, Cursor, Codex, OpenCode, Gemini CLI and others.
```bash
npx skills add adithya-s-k/FineEnvs
```
**We're looking for new end-to-end recipes** — a task, an environment, a training run, and honest results. [Contributing guide →](https://github.com/adithya-s-k/FineEnvs/blob/main/CONTRIBUTING.md)
## Citation
```bibtex
@misc{fineenvs,
author = {Kolavi, Adithya S},
title = {FineEnvs: Open Source RL Environments for LLM Agents},
year = {2026},
url = {https://github.com/adithya-s-k/FineEnvs}
}
```
|