sergiopaniego HF Staff commited on
Commit
e4d28dd
路
verified 路
1 Parent(s): c1e9830

Add 07 PortSimEnv and the Simulation RL Environments article

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -53,6 +53,7 @@ Each numbered project below is a self-contained recipe: an environment, a traini
53
  | **04** | **SmolDataEnvs** | 5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites. | [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/smoldataenvs-6ab4f2f6e09b7cb872ebc867) |
54
  | **05** | **SmolDataEnvs: Multi-harness RL** | One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints. | [Article 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl) 路 [Slides 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) 路 [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/smoldataenvs-multi-harness-rl-6abdfaaa8d74dacd481d5212) |
55
  | **06** | **Multilingual** | Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada. | [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/multilingual-multimodal-envs-6ac0c27c137f93e0799603e4) |
 
56
 
57
 
58
  # Articles & Talks
@@ -66,6 +67,7 @@ Each numbered project below is a self-contained recipe: an environment, a traini
66
  | 馃Л **The Ultimate Guide to Multi-Harness RL** | Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes. | [Read 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl) |
67
  | 馃帳 **Training a Coding Agent Through a Harness You Did Not Write** | Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO. | [Watch 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) |
68
  | 馃實 **How to turn a game into an RL environment** | The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL. | [Read 鈫抅(https://huggingface.co/spaces/FineEnvs/geoguesser-article) |
 
69
 
70
 
71
  # Environments
 
53
  | **04** | **SmolDataEnvs** | 5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites. | [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/smoldataenvs-6ab4f2f6e09b7cb872ebc867) |
54
  | **05** | **SmolDataEnvs: Multi-harness RL** | One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints. | [Article 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl) 路 [Slides 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) 路 [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/smoldataenvs-multi-harness-rl-6abdfaaa8d74dacd481d5212) |
55
  | **06** | **Multilingual** | Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada. | [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/multilingual-multimodal-envs-6ac0c27c137f93e0799603e4) |
56
+ | **07** | **PortSimEnv** | Re-plan a broken week of container-ship dockings at the Port of Barcelona, built from the port's real 2024 records and graded against a plan CP-SAT proves optimal. | [Article 鈫抅(https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) 路 [Play 鈫抅(https://huggingface.co/spaces/FineEnvs/PortSimEnv) 路 [Collection 鈫抅(https://huggingface.co/collections/FineEnvs/simulation-rl-envs) |
57
 
58
 
59
  # Articles & Talks
 
67
  | 馃Л **The Ultimate Guide to Multi-Harness RL** | Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes. | [Read 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl) |
68
  | 馃帳 **Training a Coding Agent Through a Harness You Did Not Write** | Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO. | [Watch 鈫抅(https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) |
69
  | 馃實 **How to turn a game into an RL environment** | The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL. | [Read 鈫抅(https://huggingface.co/spaces/FineEnvs/geoguesser-article) |
70
+ | 馃殺 **Simulation RL Environments** | Turning real-world work into RL environments. Part 1 builds PortSimEnv from the Port of Barcelona's 2024 records and grades every plan against a proven optimum. | [Read 鈫抅(https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) |
71
 
72
 
73
  # Environments