Spaces:
Running
Download blog.md from CoBeDigger/DataEngEnv: direct link, hf CLI and curl.
- Browser
- Download file 11.9 kB
-
https://huggingface.co/spaces/CoBeDigger/DataEngEnv/resolve/main/blog.md
- Command line
-
hf download hf://spaces/CoBeDigger/DataEngEnv/blog.md
-
curl -L -o blog.md https://huggingface.co/spaces/CoBeDigger/DataEngEnv/resolve/main/blog.md
title: >-
PipelineOps Arena: We Built an RL Environment Around the Bugs That Actually
Break Production
thumbnail: https://huggingface.co/spaces/CoBeDigger/DataEngEnv/resolve/main/imgmeta.png
authors:
- user: CoBeDigger
A 4-stage cascading ML pipeline debugger for RL agents β Meta AI Γ Hugging Face OpenEnv Hackathon 2026.
Every ML team has shipped a broken model to production at least once. The pipeline looked fine. The numbers looked good. But somewhere between the data loading and the deployment gate, something went wrong β silently, invisibly, and expensively.
We built PipelineOps Arena because these bugs are real, they happen constantly, and no one has tried to train an AI to catch them systematically.
Try it live: huggingface.co/spaces/CoBeDigger/DataEngEnv
The Four Bugs We're Testing
We didn't invent these failure modes. We've lived them.
Stage 1 β Data Repair. Someone renamed a column upstream. The script references age_years but the DataFrame column is age. KeyError. The dataset also has ~15% NaN rows that crash StandardScaler with ValueError. Two bugs, both required to pass.
Stage 2 β Training Monitor. An MLPClassifier is fed raw unscaled features β salary around 70,000 sitting next to employment years around 8. Loss diverges to NaN after epoch 1. No crash. Silent divergence.
Stage 3 β Eval Validation. scaler.fit_transform() runs on the entire dataset before train_test_split(). The model has already seen test data during normalization. It prints Accuracy: 0.98. It's lying.
Stage 4 β Deploy Gate. The model passes accuracy. It passes inference latency. But the MLOps bot measures a 0.09 demographic fairness gap across age groups β above the 0.05 deployment threshold. Deployment rejected.
These aren't abstract challenges. They are the exact class of bugs that get missed in code review because they don't crash β they just corrupt.
How the Environment Works
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β PipelineOps Arena β
β β
β Stage 1 Stage 2 Stage 3 Stage 4 β
β Data Repair β Training β Eval Fix β Deploy Gate β
β (KeyError) (NaN loss) (Leakage) (Fairness) β
β β
β Each stage: inspect β edit β run β submit β
β Grade β₯ 0.70 β advance to next stage β
β Grade < 0.70 at Stage 4 β loop back (β0.30) β
β Max steps: 60 total, 15 per stage β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
The agent talks to a FastAPI server over plain HTTP β no special SDK, just REST. Built on the OpenEnv spec.
At each step the agent gets an observation like this:
{
"current_stage": 2,
"step_number": 7,
"script_content": "...current pipeline code...",
"last_run_error": "ValueError: Input contains NaN",
"data_preview": "Shape: (500, 6)\nColumns: ['age', 'salary', ...]",
"actor_feedback": "StandardScaler not found in script.",
"stages_completed": [1],
"done": false
}
And picks one of five actions:
| Action | What it does |
|---|---|
inspect_data |
Returns dataset shape, columns, null counts, sample rows |
run_script |
Executes current pipeline in a 10-second isolated sandbox |
edit_script |
Patches specific text in the script, or replaces it entirely |
query_actor |
Asks the Code Reviewer (Stages 2β3) or MLOps Bot (Stage 4) |
submit |
Triggers the deterministic grader |
The key design decision: every reward comes from objective, code-based evaluation. No LLM-as-judge. No string matching. The grader runs the fixed script against real data and measures actual accuracy, convergence, and fairness gap. If the fix works, it works.
The Cascade Mechanic
The stages aren't just sequential β they're designed to punish shallow fixes.
Stage 4 has a loop mechanic: fail the fairness gate and the agent goes back to Stage 1 with a β0.30 penalty. Two failed loops end the episode at 0.00.
This forces genuine long-horizon planning. An agent that blindly submits at every stage will hit Stage 4, fail the fairness check, and lose everything it earned. The fairness gap (0.09 across age groups) only becomes visible when the agent explicitly queries the MLOps Bot β which runs the script in a subprocess and measures it directly. The hint is there: "Consider class_weight='balanced'". But the agent has to know to ask.
Two Reviewers the Agent Can Consult
Code Reviewer (Stages 2β3): Regex-based. Checks for data leakage patterns β whether scaler.fit() was called before or after the split. Gives natural-language feedback like "Scaler is being fit on the full dataset. This leaks test statistics into training."
MLOps Bot (Stage 4): Actually runs the agent's current script in a subprocess with a fairness-gap measurement appended. Reports accuracy, inference latency, and the demographic gap with explicit pass/fail labels. Every query_actor call re-executes β it's compute-heavy and intentional. The agent should only call it when it's ready for real feedback.
Reward Structure
Dense rewards throughout β partial credit for doing the right things, not just for completing stages.
inspect_data (first time) +0.10
edit_script: fix column name +0.20
edit_script: add dropna/clip +0.15
run_script: clean after fix +0.20
edit_script: add StandardScaler +0.30
edit_script: fix data leakage +0.30
query_actor: MLOps Bot +0.10
edit_script: add class_weight +0.30
submit: stage complete grader score [0.0β1.0]
Stage 4 failure (loop back) β0.30
Repeated action β0.03
A perfect run β all 4 stages cleared without looping β returns 1.00.
Training: Llama 3.1 8B with GRPO
We fine-tuned Llama 3.1 8B (4-bit quantized via Unsloth) using GRPO β Group Relative Policy Optimization β on Kaggle T4 GPU.
Setup: LoRA r=16, all projection layers, 0.26% trainable params, TRL GRPOTrainer.
Why Semantic Reward
The first thing we tried was episode completion as the reward signal. It produced nothing.
The untrained Llama 3.1 8B instruct model doesn't output {"action_type": "edit_script", "payload": {...}}. It outputs rich descriptive JSON β multi-step plans, code analysis, detailed reasoning about the bug. All valid ML thinking, none of it parseable as an environment action.
That meant every episode returned 0.00. reward_std = 0.0. GRPO with zero variance produces zero gradient. The model learned nothing.
So we built a semantic reward function that scores each response on format quality alone:
+0.20 Valid JSON output
+0.20 action_type key present
+0.30 Correct action keyword for current context
+0.30 Correct payload structure
This decoupled learning from episode completion. The model gets a gradient signal just for structuring its output correctly β which is the prerequisite for everything else.
Results
Left: avg group reward per training step. Right: GRPO policy loss per step. Both from a real 25-step run on Kaggle T4.
| Metric | Value |
|---|---|
| Base model | Llama 3.1 8B 4-bit (Unsloth) |
| Training steps | 25 |
| Group size (G) | 4 completions per step |
| First 5 avg reward | 0.25 |
| Last 5 avg reward | 0.38 |
| Improvement | +52% |
The reward went from 0.25 to 0.38 over 25 steps. The variance is the real signal β early steps show wide swings as the model explores different formats; later steps tighten. The model learned that structured JSON is rewarded, and started producing it more reliably.
The untrained model scored 0.00 on episode completion because it couldn't produce parseable actions. The trained adapter pushes toward the correct action format β which is the necessary first step before episode-level RL becomes possible.
Trained model: CoBeDigger/pipelineops-arena-llama3-grpo
Training notebook (Colab, re-runnable): GRPO_Training_Pipeline.ipynb
The Baseline: What Perfect Looks Like
We built a hardcoded rule-based agent that executes the optimal action sequence per stage. It completes all 4 stages in 16 steps with a score of 1.00. This validates the graders and gives a ceiling to measure against.
Stage 1: edit_script (column fix + dropna) β run_script β submit [3 steps]
Stage 2: inspect_data β edit_script (add scaler) β run_script β submit [4 steps]
Stage 3: run_script β edit_script (fix leakage) β run_script β submit [4 steps]
Stage 4: query_actor β edit_script (class_weight) β run_script β submit [4 steps]
You can watch it run live on the Space under the "Baseline vs LLM Agent" tab.
Video walkthrough: Watch on YouTube
Try It
No authentication required. Just HTTP.
# Start an episode
curl -X POST https://cobedigger-dataengenv.hf.space/reset
# Take a step
curl -X POST https://cobedigger-dataengenv.hf.space/step \
-H "Content-Type: application/json" \
-d '{"action_type": "inspect_data", "payload": {}}'
# Check pipeline progress
curl https://cobedigger-dataengenv.hf.space/pipeline_status
Or run an agent loop:
import requests
ENV = "https://cobedigger-dataengenv.hf.space"
obs = requests.post(f"{ENV}/reset").json()
for step in range(60):
action = your_agent.decide(obs)
result = requests.post(f"{ENV}/step", json=action).json()
obs = result["observation"]
reward = result["reward"]["score"]
if obs["done"]:
break
Links:
- Live Space: huggingface.co/spaces/CoBeDigger/DataEngEnv
- Trained model: CoBeDigger/pipelineops-arena-llama3-grpo
- Training notebook: Colab
- GitHub: Pranav-0710/DataEngEnv
- Video: YouTube walkthrough
What We Learned
Reward design is the hardest part. Getting a gradient signal out of a model that can't produce valid JSON yet requires rethinking what "reward" means at the start of training. Semantic reward isn't a hack β it's necessary scaffolding.
Real execution beats string matching. Every grader in this environment runs actual Python code against real data. This makes it harder to game and means reward always reflects genuine task completion.
The cascade forces planning. Agents that treat each stage independently will fail Stage 4 and lose 0.30 points. The loop mechanic rewards agents that maintain a holistic view of the pipeline β not just the current bug.
CobeDivers β Jey Pranav, Kavin Krish, Hari Meta AI Γ Hugging Face OpenEnv Hackathon 2026
