Harbor is from Laude Institute (the Terminal-Bench team), not Stanford / Snorkel AI, and the canonical repo lives at github.com/laude-institute/harbor. The "Offline batch RL only" line was wrong too, Harbor is an agent eval and RL rollout-generation framework, the official harness for Terminal-Bench 2.0, that runs autonomous agent harnesses (Claude Code, Codex CLI, OpenHands) in parallel containers via Daytona / Modal. The agent drives the loop end-to-end inside the sandbox and Harbor emits trajectories, which is a different scope from a turn-by-turn environment_factory the trainer can drive turn-by-turn.
fix(picker): include OpenEnv on the large-catalog branch
The picker's "A large catalog matters" branch labels itself as "Hub-shipped or vendor-shipped catalogs (50+ envs)", and OpenEnv's HF Spaces hub (4k+ MCP environments per the ecosystem matrix) clearly clears that bar. With the old catalog: 'small' tag, picking "large catalog" silently excluded OpenEnv even though its hub is the largest of the six. Bumping to 'large' so it surfaces alongside ORS, NeMo, Verifiers, and GEM.
- Fix the Qwen3.5 link in the scaling-progression sentence; it was pointing at qwenlm.github.io/blog/qwen3 (the Qwen3 page, wrong release). Now points at qwen.ai/blog?id=qwen3.5.
- Add a paragraph quoting the Qwen3.5 release notes directly: "extensive scaling of virtually all RL tasks and environments we could conceive", framed around their deliberate choice to raise env difficulty and generalisability rather than chase narrow benchmarks. Reinforces the article's framing that environments are the lever now, not the algorithm.
- Embed the qwen3.5_397b_a17b_scaling.png figure from qianwen-res OSS CDN as visual evidence next to that paragraph.
- Remove the Benchmax row from the "Other frameworks in the landscape" table; it sits at a different abstraction layer (distributed eval framework via SkyPilot, not a training-integrated env framework) and isn't load-bearing for the comparison.
- Repoint RL-Factory from github.com/Spydiecy/rl-factory to the actual upstream at github.com/Simple-Efficient/RL-Factory; creator label flips from "Spydiecy" to "Simple-Efficient" everywhere it appears.
- Update landscape.json data row + drop the Benchmax short-code mapping in d3-tier-map.html so the tier viz doesn't render an orphaned bubble.
- Tier-map intro line: "17 frameworks we surveyed" → "16" to stay accurate.
Audit notes were a working document during the visualisation pass; the underlying
audit decisions all landed in the actual embeds and dimension chapters and the
file no longer reflects current state, so dropping it from the repo.
chore(space): add HF Space thumbnail + update README metadata
The Space's README.md frontmatter is what HF Spaces uses to render the social card on the OUTER URL (huggingface.co/spaces/AdithyaSK/rl-environments-guide). Without a `thumbnail:` field, HF auto-generates a default card via cdn-thumbnails.huggingface.co, which is what's currently being served. The article's inner `og:image` meta tag (set via seoThumbImage) doesn't apply here because social scrapers fetch the outer URL, not the iframe.
- title: refresh to match the article's current title.
- short_description: add for the Space card byline + og:description fallback.
- thumbnail: point at the GitHub raw URL on RL_Envs_101, no binary lives in the Space repo.
- tags: add rl-environments, llm-training, reinforcement-learning so the Space surfaces in HF filters.
feat(intro): simpler title, May date, scaling-progression opener
- Title: "The ultimate guide to RL environments: building and scaling them in the LLM era". Drops the "to 1000s" and reorders so "RL environments" leads, "in the LLM era" qualifies. Reads cleaner.
- Last-updated note: April 2026 → May 2026.
- Intro opener rewrite: replace the abstract "RL is becoming central..." paragraph with a grounded scaling story. Defines an environment in concrete terms (where a model practises, gets graded, learns from interaction over long horizons), then walks the scaling progression with linked sources, Qwen3 (~20 tasks) → Qwen3-Coder (20K parallel) → MiniMax Forge / M2.5 (100K+) → Qwen3.5 (1M+ agent envs). Lands the bottleneck shift ("not 'can we set up an environment', it's 'how do we run 100,000 of them, keep them honest, and feed them into a training loop'") and keeps the bridge into anatomy.
feat(seo): point OG image at GitHub raw URL on RL_Envs_101
Avoids shipping a binary in the HF Space repo (which rejects PNG pushes outside Xet/LFS). The og:image, twitter:image, and JSON-LD image array now resolve to the absolute GitHub raw URL on the companion repo, so the social card preview pulls from there directly. Verified emitted in dist/index.html.
feat(animation): enhance animation timing and visual effects for smoother transitions
- observations.mdx: collapse from 5 + appended-closing into 4 sections (What stood out, Picking one, Framework-specific gotchas, Where this leaves us). Five sharp bullets each anchored to specific frameworks (env boundary, HTTP vs in-process, dataset coupling, reward timing, exec-backend latency) replace the looser earlier prose. Trim the standalone Scaling subsection since its non-redundant content now lives in the latency bullet, the rest is in Dimension 10.
- framework-profiles.mdx: rewrite all six profiles to a uniform shape, what it is + what ships out of the box + a "good for" line. Hyperlink each framework name to its canonical source (GitHub or homepage), with ORS getting the spec link and the openreward.ai catalog link. Drop the per-profile gotcha tails since the gotcha table in observations now owns them.
feat(hero+article): polish hero meta bar, fold closing thoughts into observations
- HeroArticle.astro: redesign Code button as black + orange-bordered split (github icon left, star icon right, no underline). Match PDF download button to the same style. Reorder cells so Code lives on the far right after Published; gate PDF cell on showPdf prop. Drop the wrap-with-button-class for the link and add :not(.repo-button) carve-outs in the underline rules.
- article.mdx: bump published to May 2 2026, set showPdf: false, fill in author + affiliation block (Adithya, Lewis, Leandro, Quentin, Amine, Ben), affiliation linked to huggingface.co.
- comparison-matrix chapter: rewrite intro and add an outro that points readers at the deployment-shape split and forwards them to the per-framework profiles.
- observations chapter: append the closing-thoughts content as a final subsection (state of the space, three takeaways, plus the "same thing under different shells" wrap).
- closing-thoughts.mdx + import + render: removed; chapter is now folded into observations.
- framework-profiles…
feat(dim-6): reframe tasks as artifact bundles, add coupling note with grounded examples
- Rewrite Dim 6 opener to actually define a task (prompt + input data + scoring contract) before listing how frameworks load it; drop the vague "feed the env a prompt" framing.
- Add "A task is more than a row" sub-section explaining that agent and coding tasks ship a bundle (file tree, test harness, setup, optional reference), not just a JSONL row. Use OpenReward SETA (task.yaml, weights.json, tests/test_outputs.py) and SWE-Bench (repo, base_commit, problem_statement, test_patch, FAIL_TO_PASS/PASS_TO_PASS) as grounded examples.
- Add "Coupling: who owns the task spec" sub-section: coupled (Verifiers Dataset spec, GEM loaders, ORS/NeMo Gym server-side, Environments Hub + OpenReward standardisation) vs BYO (OpenEnv, SkyRL Gym).
- Add note callout on data transformation, who owns the mapping when env dictates the spec, with three concrete transformation examples.
- d3-episode-control: trainer lane stop renamed "GRPO update" to "policy update" so the diagram stays algorithm-agnostic and mirrors the env lane.
- dimensions.mdx Dim 3: replace meta paragraph about the animation with three load-bearing bullets (schema portability, authoring cost, multi-turn looping universality).
- dimensions.mdx Dim 4: add a callout on what's actually inside a reward function, procedural, LLM-as-judge, dense vs sparse, orthogonal to the four architectural patterns.
- dimensions.mdx Dim 5: reframe the two control patterns as Example 1 (tool-based, trainer-driven) and Example 2 (harness-controlled, env-driven) using a coding-agent tool set, drop the per-framework bullet list in favour of the matrix embed.
- d3-reward-architecture: switch all spike animations to linear easing so the cursor and spikes stay phase-locked over long runs; the bounce already lives in the keyframe values.
feat(dim-2): refine intro/outro prose and reposition backend below env
- Dimensions 1: append a one-liner pointing readers to the RL_Envs_101
companion repo for runnable end-to-end versions of every code snippet.
- Dimension 2: convert the inline 'two patterns differ on three things'
paragraph into a bulleted list (Where it runs / What you install /
How it scales) so each point is scannable.
- Dimension 2: drop the 'Hit Play below to watch one tool call travel
through each shape' prompt; the diagram speaks for itself.
- Dimension 2: add a closing paragraph after the Slurm-cluster table
noting that most environments delegate heavy work to sandbox
providers (E2B / Modal / containers), which makes both HTTP servers
and in-process clients lightweight to scale; the deployment shape
matters for dependency isolation, not raw compute.
- d3-comm-deploy in-process panel: move the optional Backend box from
bottom-left of the training-process box to directly below the Env
card. Replace the L-shaped dashed connector with a clean vertical
drop and…
Intro / framework-inventory / why-comparison:
- Merge Abstract and Introduction into a single section with shorter,
less-repetitive prose; expand env→environment everywhere.
- Drop framework-name list from the intro paragraph; link the word
'frameworks' to the inventory chapter instead.
- Move the 'why have environment frameworks at all?' explanation into
a markdown blockquote callout at the top of the inventory chapter.
- Bold 'RL environments' on the opening line.
- Mention the companion repo (RL_Envs_101) in the intro.
- TL;DR shortened back to abstract framing.
Anatomy hero (d3-anatomy-hero.html):
- Rewrite as pure SVG (no D3) with the env on the left and the
Training Loop box on the right.
- Env contains all 9 components (Tasks, Prompt template, Initial state,
Tools / Harness, State, Execution backend, Observation, Reward, Done)
with Lucide-style icons matching the rest of the article.
- Training Loop box contains LLM Agent, Rollout, Trainer, Weight update
with vert…
feat(viz): update D3 anatomy hero styles for improved layout and readability
- Introduced a new HTML component for displaying framework specification cards, including styling and dynamic content generation via JavaScript.
- Added a global comparison matrix to facilitate side-by-side evaluation of multiple frameworks, complete with filter functionality and copy-to-clipboard feature for row data.
Add D3 latency bars and scaling comparison matrix embeds
- Introduced a new HTML embed for D3 latency bars, including styles and scripts for rendering latency visualizations and tables.
- Added a scaling comparison matrix embed with a structured table layout to compare different frameworks' capabilities and performance metrics.
feat(viz): add ecosystem matrix and effort-to-author quadrant for dim 7 & 8
Replace the markdown tables and viz placeholders in Dimensions 7
(Ecosystem & maturity) and 8 (Ease of adding new environments) with
polished embeds.
Dimension 7:
- ecosystem-matrix.html: chip-styled table with creator pills, code
chips for PyPI names, and color-tinted production-usage badges
(green for deployed, blue for platform, dashed for research).
- Visualization (d3-ecosystem-maturity) was prototyped but dropped
on user feedback; the matrix alone carries the dimension.
Dimension 8:
- d3-effort-author.html: quadrant scatter chart. X = files-to-create,
Y = concepts-to-learn, bubble size scales with time-to-first-test.
Solid stroke = ships a CLI scaffolder, dashed = start from scratch.
Median guides split easy/heavy quadrants. Hover a bubble for a
per-framework tooltip with files, concepts, time, CLI, test cmd.
In-process bubbles are jittered to avoid the (1.5,3) overlap, each
carries a 2-letter code (VF/SK/GM/OE/OR/NG) for quick scanning.
- effort-author-matrix…
Add episode control, reward architecture, and task flow matrices
- Introduced `episode-control-matrix.html` to display termination signals and loop drivers for various frameworks.
- Added `reward-arch-matrix.html` to illustrate reward architecture across different frameworks, including computation methods and verification processes.
- Created `task-flow-matrix.html` to outline task sources, loaders, prompt templates, splits, and versioning for multiple frameworks.
- Each matrix includes responsive design elements and interactive features for better user experience.
feat(viz): add comm-deploy and tool-discovery embeds for dim 2 & 3
Replace the placeholder visuals and markdown tables in Dimensions 2
(Communication & deployment) and 3 (Tool & action model) with paired
animated SVG embeds plus polished chip-tables.
Dimension 2:
- d3-comm-deploy.html: two-panel SVG (HTTP frameworks / In-process)
with Play/Pause + speed slider, animated request/response packets
showing the network hop on the HTTP side and the inline call on the
in-process side. Auto-plays on scroll-into-view.
- comm-deploy-matrix.html: chip-styled table with grouped bands
(Shape / Deployment / Trainer-side), clickable HF Spaces and
PrimeIntellect Hub links, ✓/—/N/A pill badges.
Dimension 3:
- d3-tool-discovery.html: 6 framework cards in a responsive grid,
each with a mini sequence diagram (Trainer ↔ Env) animating
discovery → invoke phases. HTTP cards show dashed wire arrows,
in-process show solid Python calls.
- tool-action-matrix.html: polished table with Kind pill (HTTP /
In-process), code chips for endpoints and decorators, sin…
feat(viz): add ship-matrix embed with table/cards/compare views
Replace the markdown component-support table in rl-environment.mdx with
a ship-matrix.html embed that offers three views:
- Table (default): emoji status (full/partial/BYO) per cell + small note
- Cards: per-framework profile with full/partial/BYO counts
- Compare: pick any two frameworks, side-by-side per-component diff
Also expand the rosetta intro with concrete vocabulary mismatches
(Rubric vs verify(), done vs terminated/truncated, task vs split row)
and tighten the support-table intro to explain the three states only.
Refactor dimensions and RL environment chapters; add code tab and concept rosetta embeds
- Updated dimensions.mdx to streamline the presentation of framework comparisons, focusing on building RL environments.
- Introduced a new code tab embed for displaying framework-specific code snippets side by side.
- Added a concept rosetta embed to clarify terminology differences across frameworks, enhancing reader comprehension.
- Removed redundant code snippets from dimensions.mdx and replaced them with dynamic embeds for better maintainability.
Adds five new D3/HTML embeds that replace ASCII or static blocks across
rl-environment.mdx and framework-inventory.mdx:
- d3-classical-rl.html: live CartPole simulation framed as Agent ↔ Environment,
with action / state·reward bus animation and slow-speed support down to 0.05x.
- d3-llm-rl-coding.html: multi-rollout coding env with synced animations,
agent (LLM policy) on top, environment (sandboxed shell) below, and the
"+ N more rollouts" placeholder pulsing only while the visible rollouts run.
- d3-rl-pipeline.html: five-stage pipeline (Tasks → Harness → Reward → Rollout
→ Trainer) with per-framework brackets above and an "RL environment"
underbrace below the first three pills.
- taxonomy-of-environment.html: components-of-an-environment view with a
grid/table toggle, default to table, header row, and per-row accent cards.
- framework-cards.html: 3x2 minimal cards for the implemented frameworks
with creator chip, transport pill, package badge, copy-to-clipboard,
rep…
feat: add anatomy hero, scaling grid banner, and inline viz placeholders
- New d3-anatomy-hero.html: LLM ↔ Env ↔ Rollout/Trainer diagram with
feedback loop, replaces the abstract placeholder.
- New banner-grid-scaling.html (set as the live hero): grid scales 0 →
max with eased motion, calmer palette, internal padding.
- 4 alternate banner variants kept for swapping (trajectories, grid,
streams, tree).
- VISUALIZATIONS_AUDIT.md catalogues the inline [Visualisation] placeholders
added across every chapter.
- Title swapped to RL-environments framing; description and abstract updated
to match.
style: replace em dashes with commas/colons across chapters
Bold-term definitions and headings get ': ', mid-sentence em dashes get
', '. Table N/A placeholders preserved via negative lookahead/lookbehind
on '|'.
Initial commit: ultimate guide to LLM environments
Research article migrated from Notion source into the
research-article-template chapter format. 9 chapters under
app/src/content/chapters covering frameworks (OpenEnv, ORS,
NeMo Gym, Verifiers, SkyRL Gym, GEM), 10 dimensions of
comparison, framework profiles, and observations.
