Blame for README.md

---

title: MaterialForge
emoji: "🔬"
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: mit
base_path: /web
---


# MaterialForge

**Inverse Autonomous Synthesis of Crystalline Atomic Lattices**

[![OpenEnv Compatible](https://img.shields.io/badge/OpenEnv-Compatible-blue)](https://openenv.org/)
[![Python 3.11](https://img.shields.io/badge/Python-3.11+-3776ab)](https://www.python.org/)
[![Docker Ready](https://img.shields.io/badge/Docker-Ready-2496ED)](https://www.docker.com/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

MaterialForge frames materials discovery as an **inverse design problem**. Given target physical properties, the agent must **discover a lattice arrangement that produces them**. The environment is built to train and evaluate LLM agents on a scientific discovery workflow rather than a static prompt task.

## Live Links

| Feature | URL |
|--|--|
| Environment API | https://huggingface.co/spaces/ArshPathan/material_forge_env |
| Interactive Dashboard | https://huggingface.co/spaces/ArshPathan/material_forge_env/playground |
| Training Blog | [BLOG.md](BLOG.md) |
| Research / Strategy Notes | [RESEARCH.md](RESEARCH.md) |
| Training Notebook | [training/MaterialForge_GRPO_Training.ipynb](training/MaterialForge_GRPO_Training.ipynb) |

## The Inverse Design Goal

The core task is:

**Given target values for hardness, conductivity, thermal resistance, and elasticity, the agent must discover a lattice arrangement that produces the target properties.**

The agent does not solve this in one shot. It must:

- observe the current 8x8 lattice state
- discover how the placement of different atom species influences global properties
- see how actions change material properties in real-time
- recover from structural instability or budget overruns
- build toward a high-quality crystalline phase that achieves the target performance

This makes MaterialForge a prime example of:

- **Theme #3.1: World Modeling**
- **Theme #2: Long-Horizon Planning**

## Why This Matters

MaterialForge is inspired by the complexity of **inverse material design**. Even though the simulation is simplified to an 8x8 lattice, the environment captures critical scientific principles:

- **Percolation pathways** for conductivity
- **Coordination-driven stability** for local structure quality
- **Phase / order formation** for crystalline vs amorphous behavior
- **Budget-constrained design** where structure discovery must be both accurate and efficient

## Training Approach

We used a GRPO-based notebook pipeline built with:

- **OpenEnv** for environment structure
- **TRL** for RL post-training
- **Unsloth** for efficient 4-bit QLoRA finetuning
- **Qwen/Qwen3-0.6B** as the base model for tool-using policy training

For the **Grand Finale Submission**, we scaled the training using a high-memory server with **~42GB VRAM**, enabling **8 generations per prompt** and real-time gradient updates for maximum stability during the discovery process.

## Training Evidence

The training results demonstrate measurable agent improvement in discovering high-performing lattice structures, achieving a **peak episodic reward of 0.86+**:

### High-Performance Setup
- **Base Model:** `Qwen/Qwen3-0.6B` (Unsloth 4-bit)
- **Generations per Prompt:** `8` (leveraging high-memory hardware for stable advantage estimation)
- **Max Completion Length:** `512` (allowing for deep scientific reasoning)
- **Hardware:** High-memory server (**~42GB VRAM**)

### Reward Curve
![Reward Curve](training/reward_curve.png)
The reward curve shows the agent successfully learning the structural tradeoffs required to reach target properties.

### Loss Curve
![Loss Curve](training/loss_curve.png)
The loss plot confirms stable policy optimization over the training steps.

### Baseline Comparison
![Baseline Comparison](training/baseline_comparison.png)
The trained agent significantly outperforms a random baseline, demonstrating a learned ability to discover valid, high-property lattice arrangements.

## Final Summary

MaterialForge demonstrates how reinforcement learning can be applied to **inverse scientific design problems**, where an agent must search the combinatorial space of structures that produce desired physical properties. By combining an OpenEnv environment with a GRPO training pipeline, we show that agents can learn to reason over lattice structures and navigate complex tradeoffs to discover optimized scientific solutions.

<div align="center">
  <p>Built for the <b>Meta PyTorch OpenEnv Hackathon</b> by Arsh Pathan</p>
</div>