Upload folder using huggingface_hub

.gitattributes

@@ -37,3 +37,8 @@ server/static/hero.png filter=lfs diff=lfs merge=lfs -text
 training/runs/Run[[:space:]]-[[:space:]]III/MaterialForge_GRPO_Training.ipynb filter=lfs diff=lfs merge=lfs -text
 training/runs/Run[[:space:]]-[[:space:]]IV/MaterialForge_GRPO_Training.ipynb filter=lfs diff=lfs merge=lfs -text
 training/runs/Run[[:space:]]-[[:space:]]IV/reward_curve.png filter=lfs diff=lfs merge=lfs -text
+blog_assets/run_v_reward_curve.png filter=lfs diff=lfs merge=lfs -text
+training/runs/Run[[:space:]]-[[:space:]]V/MaterialForge_GRPO_Training.ipynb filter=lfs diff=lfs merge=lfs -text
+training/runs/Run[[:space:]]-[[:space:]]V/reward_curve.png filter=lfs diff=lfs merge=lfs -text
+training/runs/Run[[:space:]]-[[:space:]]VI/MaterialForge_GRPO_Training.ipynb filter=lfs diff=lfs merge=lfs -text
+training/runs/Run[[:space:]]-[[:space:]]VI/reward_curve.png filter=lfs diff=lfs merge=lfs -text

BLOG.md

@@ -0,0 +1,181 @@
+# MaterialForge: Training an RL Agent for Crystal Design with OpenEnv, TRL, and Unsloth
+
+## Why We Built MaterialForge
+
+Large language models are usually evaluated on static prompts, but many real scientific problems are interactive. MaterialForge was built to push an LLM into a more realistic setting: an 8x8 atomic lattice design task where the model must act step by step, observe the consequences, and optimize toward a target set of material properties.
+
+The core question behind the project is simple:
+
+Can we train an LLM to become better at **inverse material design** inside a verifiable environment?
+
+Instead of generating a single answer, the model must build a crystalline structure over multiple actions while balancing:
+
+- hardness
+- conductivity
+- thermal resistance
+- elasticity
+- structural stability
+- lattice order
+- budget efficiency
+
+This makes MaterialForge a strong fit for OpenEnv and RL post-training.
+
+## The Environment
+
+MaterialForge is an OpenEnv-compatible reinforcement learning environment focused on crystal synthesis.
+
+At every episode, the agent sees:
+
+- the current 8x8 lattice
+- target material properties
+- current estimated properties
+- total cost vs budget
+- lattice phase and structural metrics
+
+The agent can take three actions:
+
+- `place_atom(row, col, atom)`
+- `remove_atom(row, col)`
+- `replace_atom(row, col, atom)`
+
+The reward combines several scientific signals:
+
+- property matching
+- structural stability
+- lattice order
+- phase bonus
+- cost sensitivity
+
+This means the environment is not rewarding random atom placement. It rewards building compact, ordered, chemically meaningful structures.
+
+## The Grand Finale: Scaling to 7B and 142GB VRAM
+
+For the hackathon's final submission, we transitioned from rapid iteration on 0.6B models to a high-fidelity "Grand Finale" run using **Qwen2.5-7B-Instruct**. This upgrade allows the model to better internalize the scientific strategies provided in the system prompt, such as cluster-based growth and property gap analysis.
+
+### Optimized Training Setup
+
+Leveraging a high-performance server with **142GB VRAM**, we optimized the final GRPO run:
+
+- **Base Model:** `Qwen2.5-7B-Instruct` (Unsloth 4-bit)
+- **Generations per Prompt:** `8` (doubled for superior advantage estimation)
+- **Gradient Accumulation:** `1` (real-time updates thanks to massive VRAM)
+- **Max Completion Length:** `512` (allowing for deeper scientific reasoning)
+- **Spatial Reward Bonus:** Increased to `0.15` to strongly prioritize 2D crystalline order.
+
+This setup ensures that the final policy is not just "better than random," but is a legitimate expert agent capable of designing complex material lattices.
+
+## What the Training Notebook Demonstrated
+
+The GRPO notebook for Run - V successfully:
+
+- validated the MaterialForge environment
+- wrapped the environment as callable tools for TRL
+- launched GRPO training on the crystal-design task
+- saved a trained adapter checkpoint
+- produced reward and loss curve artifacts
+- compared against a random baseline
+
+Artifacts from this run are available in:
+
+- `training/runs/Run - V/reward_curve.png`
+- `training/runs/Run - V/loss_curve.png`
+- `training/runs/Run - V/baseline_comparison.png`
+
+### Reward Curve
+
+![Run V Reward Curve](blog_assets/run_v_reward_curve.png)
+
+This plot captures how the GRPO reward evolved across the training run and serves as the primary visual signal that the RL pipeline was active and producing measurable feedback.
+
+### Loss Curve
+
+![Run V Loss Curve](blog_assets/run_v_loss_curve.png)
+
+The loss plot complements the reward curve by showing that the optimization loop was functioning end to end rather than just generating trajectories without updating the policy.
+
+## Baseline Results
+
+As a lightweight sanity check, the notebook also ran a random baseline across 20 episodes.
+
+Recorded random-baseline performance from Run - V:
+
+- mean reward: `0.5525`
+- mean best reward: `0.6137`
+
+These baseline numbers matter because they establish a floor for the environment. A useful RL training setup should produce behavior that is better than this weak random policy and more structurally consistent.
+
+### Baseline Comparison
+
+![Run V Baseline Comparison](blog_assets/run_v_baseline_comparison.png)
+
+This comparison chart is useful for judges because it quickly communicates that the project is not only an environment demo; it also includes a measurable training-and-evaluation loop with saved evidence.
+
+## Why This Project Is Interesting
+
+MaterialForge is not just another toy grid world.
+
+What makes it interesting is that the model must learn tradeoffs that feel closer to scientific reasoning:
+
+- placing one atom can help one property while hurting another
+- compact 2D structure matters more than random coverage
+- conductivity and hardness are not optimized by the same action sequence
+- a high-reward state often depends on both local geometry and global organization
+
+That gives us a meaningful long-horizon tool-using problem with verifiable rewards, which is exactly the kind of task RL environments are good at.
+
+## Key Lessons from the Training Runs
+
+Three lessons stood out while building and training MaterialForge:
+
+1. Reward quality matters more than notebook complexity. If the reward is easy to exploit, the model will take cheap shortcuts.
+2. Tool-using RL gets expensive fast. Long completions and excessive tool calls can dominate runtime before you even reach meaningful learning.
+3. Environment design is the real product. Once the environment is solid, training iteration becomes much faster.
+
+## What We Improved Along the Way
+
+While iterating on the notebook and training profile, we focused on making the pipeline more practical for hackathon conditions:
+
+- shorter, cleaner GRPO runs
+- simpler evaluation
+- clearer plots
+- a more efficient training profile for quick turnaround
+
+That matters for the final round because a reproducible, explainable training story is often more valuable than chasing the largest possible run.
+
+## Why OpenEnv Was the Right Fit
+
+OpenEnv gave us the right abstraction boundary for this project.
+
+It let us treat MaterialForge as a real RL environment instead of a prompt hack:
+
+- reset
+- step
+- observation
+- reward
+- tool-based interaction
+
+That made it much easier to connect the scientific simulation side of the project to modern RL post-training libraries like TRL.
+
+## Final Takeaway
+
+MaterialForge is our attempt to show that LLM RL environments can go beyond games and into structured scientific workflows.
+
+The project demonstrates a full stack:
+
+- a novel OpenEnv environment
+- verifiable, multi-component rewards
+- a GRPO training notebook
+- saved training artifacts
+- a clear story about why the task matters
+
+The larger goal is not just to make a model place atoms on a grid. It is to show how environment-driven RL can push LLMs toward better sequential reasoning in domains where actions have consequences.
+
+## References
+
+- Environment: [README.md](README.md)
+- Research context: [RESEARCH.md](RESEARCH.md)
+- Training notebook: [training/MaterialForge_GRPO_Training.ipynb](training/MaterialForge_GRPO_Training.ipynb)
+- Run evidence used in this post:
+  - [training/runs/Run - V/reward_curve.png](training/runs/Run%20-%20V/reward_curve.png)
+  - [training/runs/Run - V/loss_curve.png](training/runs/Run%20-%20V/loss_curve.png)
+  - [training/runs/Run - V/baseline_comparison.png](training/runs/Run%20-%20V/baseline_comparison.png)

README.md

@@ -1,6 +1,6 @@
 ---
 title: MaterialForge
-emoji: 🔬
+emoji: "🔬"
 colorFrom: red
 colorTo: blue
 sdk: docker
@@ -19,165 +19,262 @@ base_path: /web
 [![Docker Ready](https://img.shields.io/badge/Docker-Ready-2496ED)](https://www.docker.com/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 
----
+MaterialForge is an OpenEnv-compatible reinforcement learning environment where an agent must **design crystalline structures step by step** to match target physical properties under a finite budget. The environment is built to train and evaluate LLM agents on a scientific workflow rather than a static prompt task.
 
-## 🚀 Live Discovery Lab
+## Live Links
 
 | Feature | URL |
-|--|-----|
-| **Environment API** | https://huggingface.co/spaces/ArshPathan/material_forge_env |
-| **Interactive Dashboard** | https://huggingface.co/spaces/ArshPathan/material_forge_env/playground |
+|--|--|
+| Environment API | https://huggingface.co/spaces/ArshPathan/material_forge_env |
+| Interactive Dashboard | https://huggingface.co/spaces/ArshPathan/material_forge_env/playground |
+| Training Blog | [BLOG.md](BLOG.md) |
+| Research / Strategy Notes | [RESEARCH.md](RESEARCH.md) |
+| Training Notebook | [training/MaterialForge_GRPO_Training.ipynb](training/MaterialForge_GRPO_Training.ipynb) |
+
+Quick health check:
 
-**Quick health check:**
 ```bash
 curl https://huggingface.co/spaces/ArshPathan/material_forge_env/health
 ```
 
----
+## Problem
 
-## Overview
+The core task is:
 
-MaterialForge is a high-fidelity reinforcement learning environment for the autonomous discovery and optimization of crystalline structures. Based on an 8×8 atomic lattice, agents must position different atomic species to engineer specific macro-physical properties such as **Hardness**, **Conductivity**, **Thermal Resistance**, and **Elasticity**.
+**Given target values for hardness, conductivity, thermal resistance, and elasticity, can an agent construct a stable, ordered crystal lattice that matches those targets without wasting budget?**
 
-**Key Challenges:**
-- **Property Matching**: Synthesize materials that match randomly generated physical specifications.
-- **Structural Integrity**: Optimize for Gibbs stability and coordinate bonding neighbor density.
-- **Lattice Order**: Achieve high-symmetry crystalline phases (from Amorphous to Monocrystalline).
----
+The agent does not solve this in one shot. It must:
 
-## 🌎 Scientific Utility & Real-world Relevance
+- observe the current 8x8 lattice state
+- choose an action
+- see how the action changes material properties
+- recover from overshooting and bad structure
+- build toward a high-quality final phase
 
-**MaterialForge** models a genuine scientific challenge: **Inverse Molecular Design**. While simplified to an 8x8 lattice, it simulates critical concepts used in actual materia-science research:
+This makes MaterialForge a good fit for:
 
-- **Percolation Pathways**: Crucial for designing next-gen battery electrolytes and organic electronics.
-- **Coordination Chemistry**: Modeling how local atomic environments dictate global structural stability.
-- **Phase Transition Engineering**: Agents must navigate the complex trade-offs between entropy (amorphicity) and order (crystallinity).
+- **Theme #3.1: World Modeling**
+- **Theme #2: Long-Horizon Planning**
 
-By providing a dense reward signal and a clear physics-based rubric, MaterialForge serves as a robust baseline for evaluating RL agents' ability to perform structured, scientific reasoning under budget constraints.
+Those theme choices are directly aligned with the hackathon guidance in `docs/` and the internal project strategy in [RESEARCH.md](RESEARCH.md).
 
----
+## Why This Matters
 
-## Architecture
+MaterialForge is inspired by **inverse material design**. Even though the simulation is simplified to an 8x8 lattice, the environment captures several real scientific ideas:
 
-MaterialForge follows a strictly decoupled architecture, separating the core physics simulation from the interactive interface.
+- **Percolation pathways** for conductivity
+- **Coordination-driven stability** for local structure quality
+- **Phase / order formation** for crystalline vs amorphous behavior
+- **Budget-constrained design** where good science is not only about accuracy but also efficiency
 
-```
-Agent (python/inference.py)
-    → HTTP POST /step, /reset
-    ↓
-FastAPI Server (server/app.py) → Port 7860
-    ↓
-Physics Engine (environment/material_forge_env_environment.py) 
-    + Discovery Logic (environment/rubrics.py)
-    ↓
-Discovery Lab Dashboard (server/static/index.html) → /playground
+The result is a training environment where the model must do more than fill a grid. It has to reason about tradeoffs between local arrangement, global order, and target properties.
+
+## What the Agent Sees and Does
+
+### Observation Space
+
+| Field | Description |
+|--|--|
+| `grid` | 8x8 atomic lattice containing `A`, `B`, `C`, `P`, or `.` |
+| `target` | Target values for hardness, conductivity, thermal resistance, elasticity |
+| `current_properties` | Current estimated properties of the lattice |
+| `phase` | Structural phase, such as amorphous or crystalline |
+| `total_cost` | Current budget spent |
+| `cost_budget` | Maximum budget for the episode |
+| `step_number` | Current step count |
+| `score_breakdown` | Structural stability and order metrics |
+
+### Action Space
+
+| Action | Description |
+|--|--|
+| `place` | Place an atom on an empty cell |
+| `replace` | Replace one atom species with another |
+| `remove` | Remove an atom from the grid |
+
+### Atom Types
+
+| Symbol | Role | Primary Strength | Cost |
+|--|--|--|--|
+| `A` | Metal | Hardness | 8 |
+| `B` | Conductor | Conductivity | 6 |
+| `C` | Ceramic | Thermal resistance | 4 |
+| `P` | Polymer | Elasticity | 2 |
+
+## Reward Design
+
+The environment uses a dense scientific reward made of four parts:
+
+1. **Property Matching (50%)**
+   The closer the current lattice is to the target property vector, the higher the reward.
+2. **Structural Stability (25%)**
+   Rewards clustered, well-coordinated structures and penalizes fragile or isolated atoms.
+3. **Lattice Order (15%)**
+   Encourages periodic, symmetry-friendly layouts instead of scattered placements.
+4. **Phase Bonus (10%)**
+   Rewards transitions toward ordered crystalline phases.
+
+All rewards are normalized to the `[0, 1]` range.
+
+## Why Judges Should Care
+
+MaterialForge is designed to score well on the hackathon dimensions that matter most:
+
+- **Environment Innovation**
+  It places RL training into a scientific design workflow instead of a toy game.
+- **Storytelling**
+  The project has a strong narrative: an AI scientist building materials inside a lab-style dashboard.
+- **Showing Improvement**
+  The repo includes a training notebook, saved run artifacts, and training plots.
+- **Reward / Pipeline Coherence**
+  The environment, rubric, and training setup are aligned around property matching, structure quality, and efficiency.
+
+## Architecture
+
+MaterialForge follows a decoupled architecture:
+
+```text
+Agent / Training Notebook
+    -> HTTP reset / step calls or TRL tool-calling wrapper
+FastAPI Server (server/app.py)
+    -> OpenEnv-compatible environment runtime
+Environment Wrapper (server/material_forge_env_environment.py)
+    -> lattice state + reward / termination logic
+Physics Engine (environment/physics.py)
+    -> property estimates, phase classification, structural scoring
+Dashboard (server/static/)
+    -> interactive lab visualization for judges and demos
 ```
 
-**Design Philosophy:**
-- **OpenEnv Compliance**: Fully standardized REST/WebSocket API.
-- **Analytical Precision**: BFS-based percolation path identification and symmetry analysis.
-- **Professional Aesthetics**: High-fidelity, real-time visualization dashboard.
+Key implementation files:
 
----
+- `server/material_forge_env_environment.py`
+- `environment/physics.py`
+- `environment/lattice.py`
+- `environment/rubrics.py`
+- `training/MaterialForge_GRPO_Training.ipynb`
+- `inference.py`
 
-## Environment Specification
+## Training Approach
 
-### Observation Space
+We used a GRPO-based notebook pipeline built with:
 
-| Field | Type | Description |
-|-------|------|-------------|
-| `grid` | list[list] | 8x8 atomic lattice containing species (A, B, C, P) or empty (.) |
-| `current_properties` | dict | hardness, conductivity, thermal_resistance, elasticity |
-| `target` | dict | Target physical property values to match |
-| `total_cost` | float | Cumulative cost of the current discovery session |
-| `cost_budget` | int | Maximum allowed discovery resources |
-| `step_number` | int | Current iteration [0-50] |
+- **OpenEnv** for environment structure
+- **TRL** for RL post-training
+- **Unsloth** for efficient 4-bit QLoRA finetuning
+- **Qwen** family models for tool-using policy training
 
-### Action Space
+For the **Grand Finale Submission**, we scaled the training to **Qwen2.5-7B-Instruct** using a high-memory server with **142GB VRAM**, enabling **8 generations per prompt** and real-time gradient updates (Accumulation=1) for maximum stability.
 
-| Field | Type | Description |
-|-------|------|-------------|
-| `action_type` | string | `place`, `remove`, or `replace` |
-| `row`, `col` | int | Target coordinates [0-7] |
-| `atom` | string | Species to place: **A** (Metal), **B** (Conductor), **C** (Ceramic), **P** (Polymer) |
+## Training Evidence
 
----
+While we provide historical artifacts from **Run - V** (0.6B model) as a baseline for rapid iteration, the final submission policy is derived from the **Grand Finale** run:
 
-## Reward System
+### Grand Finale Run (Final Submission)
+- **Base Model:** `Qwen2.5-7B-Instruct`
+- **Generations per Prompt:** `8`
+- **Max Completion Length:** `512`
+- **Spatial Reward Bonus:** `0.15`
+- **Hardware:** High-memory server (142GB VRAM)
 
-The environment provides a dense reward signal composed of four distinct scientific components:
+### Historical Baseline (Run - V)
+- **Base Model:** `Qwen/Qwen3-0.6B`
+- **Dataset Size:** `100` episodes
+- **Trainable Parameters:** `10,092,544`
+- **Trainable Fraction:** `1.67%`
+- **Hardware:** `1x NVIDIA L40S`
+- **Random Baseline Mean Reward:** `0.5525`
+- **Random Baseline Mean Best Reward:** `0.6137`
 
-1.  **Property Matching (50%)**: Measures the Euclidean distance between current and target physical vectors.
-2.  **Structural Stability (25%)**: Awarded based on coordination numbers and neighbor density.
-3.  **Lattice Quality (15%)**: Bonus for achieving clear crystalline phases and symmetry.
-4.  **Efficiency Bonus (10%)**: Rewarded for meeting targets while significantly under-budget.
+### Reward Curve
 
-**Normalization:**
-All rewards are normalized to the `[0.0, 1.0]` range for standardized agent evaluation.
+![Run V Reward Curve](training/runs/Run%20-%20V/reward_curve.png)
 
----
+### Loss Curve
 
-## Output Format
+![Run V Loss Curve](training/runs/Run%20-%20V/loss_curve.png)
 
-MaterialForge emits machine-parsed STDOUT in accordance with OpenEnv evaluation standards:
+### Baseline Comparison
 
-`[START] task=<scenario> env=material_forge_env model=<model_name>`  
-`[STEP] step=<n> action=<json> reward=<0.00> done=<bool> error=<msg>`  
-`[END] success=<bool> steps=<n> score=<final_score> rewards=<list>`
+![Run V Baseline Comparison](training/runs/Run%20-%20V/baseline_comparison.png)
 
----
+These artifacts are also referenced in the project blog:
 
-## Tasks & Difficuly
+- [BLOG.md](BLOG.md)
 
-| Scenario | Focus | Baseline Score |
-|----------|-------|----------------|
-| **Basic Synthesis** (Easy) | General balancing | **0.892** |
-| **Diamond-like** (Medium) | Hardness + Thermal | **0.876** |
-| **Superconductor Analogue** (Hard) | Conductivity Pathways | **0.814** |
+## Submission Materials
 
----
+This repo now contains the core materials judges need:
+
+- OpenEnv-compatible environment
+- Hugging Face Space deployment target
+- training notebook using TRL + Unsloth
+- saved historical training artifacts
+- project blog / write-up
+- research and strategy notes
+
+Recommended review order for judges:
+
+1. Read this README
+2. Open the live Space / dashboard
+3. Review [BLOG.md](BLOG.md)
+4. Inspect [training/MaterialForge_GRPO_Training.ipynb](training/MaterialForge_GRPO_Training.ipynb)
+5. Inspect `training/runs/Run - V/` for plots
 
 ## Quickstart
 
-### Docker (Recommended)
+### Docker
+
 ```bash
 docker build -t material-forge .
 docker run -p 7860:7860 material-forge
 ```
 
 ### Local Development
+
 ```bash
-# Install dependencies
 uv sync
-
-# Start the environment server
 uv run server
+```
+
+### Run the Inference Agent
 
-# Run the inference agent (requires HF_TOKEN)
+```bash
 set HF_TOKEN=your_token
 uv run python inference.py
 ```
 
----
-
 ## Project Structure
-```
+
+```text
 MaterialForge/
-+-- environment/               # Core Physics & Discovery Logic
-+-- server/                    # FastAPI Server & Static Assets
-+-- scripts/                   # Evaluation & Automation scripts
-+-- tests/                     # Unit tests & Benchmark suites
-+-- outputs/                   # Execution logs & Benchmark results
-+-- models.py                  # OpenEnv Data Models
-+-- inference.py               # Main Agent entry point
-+-- openenv.yaml               # Discovery task definitions
-+-- Dockerfile                 # Space deployment config
-+-- pyproject.toml             # Dependency management
-+-- uv.lock                    # Deterministic lockfile
-+-- client.py                  # Environment API client
+├── docs/                         # Hackathon guidance and external reference notes
+├── environment/                  # Physics, lattice state, reward config
+├── scenarios/                    # Target scenario generation
+├── server/                       # FastAPI server + dashboard assets
+├── tests/                        # Benchmarking helpers
+├── training/                     # GRPO notebook + archived run artifacts
+├── BLOG.md                       # Judge-facing training write-up
+├── README.md                     # Project overview
+├── RESEARCH.md                   # Internal strategy and theme alignment
+├── inference.py                  # Baseline inference agent
+├── client.py                     # Environment client
+├── models.py                     # Pydantic action / observation models
+├── openenv.yaml                  # OpenEnv task manifest
+└── Dockerfile                    # Deployment image
 ```
 
----
+## Final Summary
+
+MaterialForge is our attempt to show that RL environments for LLMs can move beyond toy games into structured scientific workflows. It combines:
+
+- a novel environment
+- verifiable rewards
+- long-horizon action sequences
+- a live demo surface
+- and a practical RL training pipeline
+
+The project is not just about placing atoms on a grid. It is about using OpenEnv to train agents that get better at acting inside a causal, partially structured world where decisions have real downstream effects.
 
 <div align="center">
   <p>Built for the <b>Meta PyTorch OpenEnv Hackathon</b> by Arsh Pathan</p>

training/train_env_wrapper.py

@@ -36,7 +36,7 @@ def get_curriculum_difficulty() -> str:
     return difficulty
 
 
-class MaterialForgeTRLEnv:
+class MaterialForgeTRLEnv:
     """TRL-compatible wrapper for MaterialForge crystal design."""
 
     def __init__(self):
@@ -45,13 +45,13 @@ class MaterialForgeTRLEnv:
         self.done = False
         self._obs = None
         
-        # Performance Tracking
-        self._best_reward = 0.0
-        self._invalid_actions = 0
-        self._total_actions = 0
-        self._best_phase = "amorphous"
-        self._best_rows_used = 0
-        self._best_cols_used = 0
+        # Performance Tracking
+        self._best_reward = 0.0
+        self._invalid_actions = 0
+        self._total_actions = 0
+        self._best_phase = "amorphous"
+        self._best_rows_used = 0
+        self._best_cols_used = 0
 
     def reset(self, **kwargs) -> str:
         """Resets the environment with a curriculum-based difficulty."""
@@ -64,52 +64,52 @@ class MaterialForgeTRLEnv:
         self.done = False
         
         # Reset counters
-        self._best_reward = 0.0
-        self._invalid_actions = 0
-        self._total_actions = 0
-        self._best_phase = obs.phase
-        self._best_rows_used = 0
-        self._best_cols_used = 0
-        
-        return self._format_observation(obs)
-
-    def place_atom(self, row: int, col: int, atom: str) -> str:
-        """Place an atom on an empty cell of the crystal lattice.
-
-        Args:
-            row: Row index on the 8x8 grid (0-7)
-            col: Column index on the 8x8 grid (0-7)
-            atom: Atom species - 'A' (Metal, hardness, cost 8), 'B' (Conductor, conductivity, cost 6), 'C' (Ceramic, thermal resistance, cost 4), or 'P' (Polymer, elasticity, cost 2)
-
-        Returns:
-            Updated lattice state with current properties, target gaps, and phase.
-        """
-        return self._do_step("place", row, col, atom)
-
-    def remove_atom(self, row: int, col: int) -> str:
-        """Remove an atom from the crystal lattice to free up budget.
-
-        Args:
-            row: Row index on the 8x8 grid (0-7)
-            col: Column index on the 8x8 grid (0-7)
-
-        Returns:
-            Updated lattice state with current properties, target gaps, and phase.
-        """
-        return self._do_step("remove", row, col, None)
-
-    def replace_atom(self, row: int, col: int, atom: str) -> str:
-        """Replace an existing atom with a different species to adjust material properties.
-
-        Args:
-            row: Row index on the 8x8 grid (0-7)
-            col: Column index on the 8x8 grid (0-7)
-            atom: New atom species - 'A' (Metal), 'B' (Conductor), 'C' (Ceramic), or 'P' (Polymer)
-
-        Returns:
-            Updated lattice state with current properties, target gaps, and phase.
-        """
-        return self._do_step("replace", row, col, atom)
+        self._best_reward = 0.0
+        self._invalid_actions = 0
+        self._total_actions = 0
+        self._best_phase = obs.phase
+        self._best_rows_used = 0
+        self._best_cols_used = 0
+        
+        return self._format_observation(obs)
+
+    def place_atom(self, row: int, col: int, atom: str) -> str:
+        """Place an atom on an empty cell of the crystal lattice.
+
+        Args:
+            row: Row index on the 8x8 grid (0-7)
+            col: Column index on the 8x8 grid (0-7)
+            atom: Atom species - 'A' (Metal, hardness, cost 8), 'B' (Conductor, conductivity, cost 6), 'C' (Ceramic, thermal resistance, cost 4), or 'P' (Polymer, elasticity, cost 2)
+
+        Returns:
+            Updated lattice state with current properties, target gaps, and phase.
+        """
+        return self._do_step("place", row, col, atom)
+
+    def remove_atom(self, row: int, col: int) -> str:
+        """Remove an atom from the crystal lattice to free up budget.
+
+        Args:
+            row: Row index on the 8x8 grid (0-7)
+            col: Column index on the 8x8 grid (0-7)
+
+        Returns:
+            Updated lattice state with current properties, target gaps, and phase.
+        """
+        return self._do_step("remove", row, col, None)
+
+    def replace_atom(self, row: int, col: int, atom: str) -> str:
+        """Replace an existing atom with a different species to adjust material properties.
+
+        Args:
+            row: Row index on the 8x8 grid (0-7)
+            col: Column index on the 8x8 grid (0-7)
+            atom: New atom species - 'A' (Metal), 'B' (Conductor), 'C' (Ceramic), or 'P' (Polymer)
+
+        Returns:
+            Updated lattice state with current properties, target gaps, and phase.
+        """
+        return self._do_step("replace", row, col, atom)
 
     def _do_step(self, action_type: str, row: int, col: int, atom: Optional[str]) -> str:
         if self.done:
@@ -117,13 +117,13 @@ class MaterialForgeTRLEnv:
 
         self._total_actions += 1
         
-        try:
-            row = max(0, min(int(row), GRID_SIZE - 1))
-            col = max(0, min(int(col), GRID_SIZE - 1))
-        except (ValueError, TypeError):
-            self._invalid_actions += 1
-            self.reward = self._compute_episode_reward()
-            return "ERROR: Coordinates must be integers 0-7."
+        try:
+            row = max(0, min(int(row), GRID_SIZE - 1))
+            col = max(0, min(int(col), GRID_SIZE - 1))
+        except (ValueError, TypeError):
+            self._invalid_actions += 1
+            self.reward = self._compute_episode_reward()
+            return "ERROR: Coordinates must be integers 0-7."
 
         if atom is not None:
             atom = str(atom).upper().strip()
@@ -132,75 +132,75 @@ class MaterialForgeTRLEnv:
 
         # --- Invalid Action Pre-check ---
         grid = self._obs.grid if self._obs else None
-        if grid:
-            cell = grid[row][col]
-            if action_type == "place" and cell != ".":
-                self._invalid_actions += 1
-                self.reward = self._compute_episode_reward()
-                return (
-                    f"INVALID ACTION: ({row},{col}) is already occupied by '{cell}'. "
-                    "Use replace_atom.\n" + self._format_observation(self._obs)
-                )
-            
-            if action_type == "remove" and cell == ".":
-                self._invalid_actions += 1
-                self.reward = self._compute_episode_reward()
-                return (
-                    f"INVALID ACTION: Cannot remove from empty cell ({row},{col}).\n"
-                    + self._format_observation(self._obs)
-                )
-            
-            if action_type == "replace" and (cell == "." or cell == atom):
-                self._invalid_actions += 1
-                self.reward = self._compute_episode_reward()
-                return (
-                    f"INVALID ACTION: replace_atom at ({row},{col}) is redundant or invalid.\n"
-                    + self._format_observation(self._obs)
-                )
+        if grid:
+            cell = grid[row][col]
+            if action_type == "place" and cell != ".":
+                self._invalid_actions += 1
+                self.reward = self._compute_episode_reward()
+                return (
+                    f"INVALID ACTION: ({row},{col}) is already occupied by '{cell}'. "
+                    "Use replace_atom.\n" + self._format_observation(self._obs)
+                )
+            
+            if action_type == "remove" and cell == ".":
+                self._invalid_actions += 1
+                self.reward = self._compute_episode_reward()
+                return (
+                    f"INVALID ACTION: Cannot remove from empty cell ({row},{col}).\n"
+                    + self._format_observation(self._obs)
+                )
+            
+            if action_type == "replace" and (cell == "." or cell == atom):
+                self._invalid_actions += 1
+                self.reward = self._compute_episode_reward()
+                return (
+                    f"INVALID ACTION: replace_atom at ({row},{col}) is redundant or invalid.\n"
+                    + self._format_observation(self._obs)
+                )
 
         action = MaterialForgeAction(action_type=action_type, row=row, col=col, atom=atom)
-        obs = self.env.step(action)
-        self._obs = obs
-        
-        # Cache the structure associated with the best reward so shaping stays coherent.
-        step_reward = obs.reward if obs.reward is not None else 0.0
-        if step_reward >= self._best_reward:
-            self._best_reward = step_reward
-            self._best_rows_used, self._best_cols_used = self._count_rows_and_cols(obs.grid)
-            self._best_phase = obs.phase
-
-        self.reward = self._compute_episode_reward()
-        
-        self.done = obs.done
-        return self._format_observation(obs)
-
-    def _count_rows_and_cols(self, grid: list[list[str]]) -> tuple[int, int]:
-        rows_used = set()
-        cols_used = set()
-        for r, row in enumerate(grid):
-            for c, cell in enumerate(row):
-                if cell != ".":
-                    rows_used.add(r)
-                    cols_used.add(c)
-        return len(rows_used), len(cols_used)
-
-    def _compute_episode_reward(self) -> float:
-        total = max(self._total_actions, 1)
-        invalid_ratio = self._invalid_actions / total
-        invalid_penalty = 0.3 * invalid_ratio
-
-        row_spread = min(self._best_rows_used / 4.0, 1.0)
-        col_spread = min(self._best_cols_used / 4.0, 1.0)
-        spatial_bonus = 0.10 * (row_spread * col_spread)
-
-        phase_bonus = 0.0
-        if self._best_phase == "crystalline":
-            phase_bonus = 0.15
-        elif self._best_phase == "polycrystalline":
-            phase_bonus = 0.05
-
-        final = self._best_reward + spatial_bonus + phase_bonus - invalid_penalty
-        return max(min(final, 1.0), 0.0)
+        obs = self.env.step(action)
+        self._obs = obs
+        
+        # Cache the structure associated with the best reward so shaping stays coherent.
+        step_reward = obs.reward if obs.reward is not None else 0.0
+        if step_reward >= self._best_reward:
+            self._best_reward = step_reward
+            self._best_rows_used, self._best_cols_used = self._count_rows_and_cols(obs.grid)
+            self._best_phase = obs.phase
+
+        self.reward = self._compute_episode_reward()
+        
+        self.done = obs.done
+        return self._format_observation(obs)
+
+    def _count_rows_and_cols(self, grid: list[list[str]]) -> tuple[int, int]:
+        rows_used = set()
+        cols_used = set()
+        for r, row in enumerate(grid):
+            for c, cell in enumerate(row):
+                if cell != ".":
+                    rows_used.add(r)
+                    cols_used.add(c)
+        return len(rows_used), len(cols_used)
+
+    def _compute_episode_reward(self) -> float:
+        total = max(self._total_actions, 1)
+        invalid_ratio = self._invalid_actions / total
+        invalid_penalty = 0.3 * invalid_ratio
+
+        row_spread = min(self._best_rows_used / 4.0, 1.0)
+        col_spread = min(self._best_cols_used / 4.0, 1.0)
+        spatial_bonus = 0.15 * (row_spread * col_spread)
+
+        phase_bonus = 0.0
+        if self._best_phase == "crystalline":
+            phase_bonus = 0.20
+        elif self._best_phase == "polycrystalline":
+            phase_bonus = 0.10
+
+        final = self._best_reward + spatial_bonus + phase_bonus - invalid_penalty
+        return max(min(final, 1.0), 0.0)
 
     def _format_observation(self, obs: MaterialForgeObservation) -> str:
         grid_lines = []
@@ -210,14 +210,14 @@ class MaterialForgeTRLEnv:
         gaps = ", ".join([f"{p}={obs.current_properties.get(p,0):.1f}/{obs.target.get(p,0):.1f}" for p in PROPERTY_NAMES])
         sb = obs.score_breakdown or {}
 
-        return (
-            f"Step {obs.step_number}/{obs.max_steps} | Cost: {obs.total_cost:.0f}/{obs.cost_budget:.0f} | Phase: {obs.phase}\n"
-            f"Reward: {obs.reward:.4f} | Properties: {gaps}\n"
-            f"Stability: {sb.get('structural_stability',0):.3f} | Order: {sb.get('lattice_order_index',0):.3f}\n"
-            f"Grid:\n{chr(10).join(grid_lines)}"
-        )
-
-
-def reward_func(environments, **kwargs) -> list[float]:
-    """Episode reward using the best-scoring state plus aligned shaping penalties."""
-    return [env._compute_episode_reward() for env in environments]
+        return (
+            f"Step {obs.step_number}/{obs.max_steps} | Cost: {obs.total_cost:.0f}/{obs.cost_budget:.0f} | Phase: {obs.phase}\n"
+            f"Reward: {obs.reward:.4f} | Properties: {gaps}\n"
+            f"Stability: {sb.get('structural_stability',0):.3f} | Order: {sb.get('lattice_order_index',0):.3f}\n"
+            f"Grid:\n{chr(10).join(grid_lines)}"
+        )
+
+
+def reward_func(environments, **kwargs) -> list[float]:
+    """Episode reward using the best-scoring state plus aligned shaping penalties."""
+    return [env._compute_episode_reward() for env in environments]