Spaces:
Sleeping
Sleeping
| title: MaterialForge | |
| emoji: "🔬" | |
| colorFrom: red | |
| colorTo: blue | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| license: mit | |
| base_path: /web | |
| # MaterialForge | |
| **Inverse Autonomous Synthesis of Crystalline Atomic Lattices** | |
| [](https://openenv.org/) | |
| [](https://www.python.org/) | |
| [](https://www.docker.com/) | |
| [](LICENSE) | |
| MaterialForge frames materials discovery as an **inverse design problem**. Given target physical properties, the agent must **discover a lattice arrangement that produces them**. The environment is built to train and evaluate LLM agents on a scientific discovery workflow rather than a static prompt task. | |
| ## Live Links | |
| | Feature | URL | | |
| |--|--| | |
| | Environment API | https://huggingface.co/spaces/ArshPathan/material_forge_env | | |
| | Interactive Dashboard | https://huggingface.co/spaces/ArshPathan/material_forge_env/playground | | |
| | Training Blog | [BLOG.md](BLOG.md) | | |
| | Research / Strategy Notes | [RESEARCH.md](RESEARCH.md) | | |
| | Training Notebook | [training/MaterialForge_GRPO_Training.ipynb](training/MaterialForge_GRPO_Training.ipynb) | | |
| ## The Inverse Design Goal | |
| The core task is: | |
| **Given target values for hardness, conductivity, thermal resistance, and elasticity, the agent must discover a lattice arrangement that produces the target properties.** | |
| The agent does not solve this in one shot. It must: | |
| - observe the current 8x8 lattice state | |
| - discover how the placement of different atom species influences global properties | |
| - see how actions change material properties in real-time | |
| - recover from structural instability or budget overruns | |
| - build toward a high-quality crystalline phase that achieves the target performance | |
| This makes MaterialForge a prime example of: | |
| - **Theme #3.1: World Modeling** | |
| - **Theme #2: Long-Horizon Planning** | |
| ## Why This Matters | |
| MaterialForge is inspired by the complexity of **inverse material design**. Even though the simulation is simplified to an 8x8 lattice, the environment captures critical scientific principles: | |
| - **Percolation pathways** for conductivity | |
| - **Coordination-driven stability** for local structure quality | |
| - **Phase / order formation** for crystalline vs amorphous behavior | |
| - **Budget-constrained design** where structure discovery must be both accurate and efficient | |
| ## Training Approach | |
| We used a GRPO-based notebook pipeline built with: | |
| - **OpenEnv** for environment structure | |
| - **TRL** for RL post-training | |
| - **Unsloth** for efficient 4-bit QLoRA finetuning | |
| - **Qwen/Qwen3-0.6B** as the base model for tool-using policy training | |
| For the **Grand Finale Submission**, we scaled the training using a high-memory server with **~42GB VRAM**, enabling **8 generations per prompt** and real-time gradient updates for maximum stability during the discovery process. | |
| ## Training Evidence | |
| The training results demonstrate measurable agent improvement in discovering high-performing lattice structures, achieving a **peak episodic reward of 0.86+**: | |
| ### High-Performance Setup | |
| - **Base Model:** `Qwen/Qwen3-0.6B` (Unsloth 4-bit) | |
| - **Generations per Prompt:** `8` (leveraging high-memory hardware for stable advantage estimation) | |
| - **Max Completion Length:** `512` (allowing for deep scientific reasoning) | |
| - **Hardware:** High-memory server (**~42GB VRAM**) | |
| ### Reward Curve | |
|  | |
| The reward curve shows the agent successfully learning the structural tradeoffs required to reach target properties. | |
| ### Loss Curve | |
|  | |
| The loss plot confirms stable policy optimization over the training steps. | |
| ### Baseline Comparison | |
|  | |
| The trained agent significantly outperforms a random baseline, demonstrating a learned ability to discover valid, high-property lattice arrangements. | |
| ## Final Summary | |
| MaterialForge demonstrates how reinforcement learning can be applied to **inverse scientific design problems**, where an agent must search the combinatorial space of structures that produce desired physical properties. By combining an OpenEnv environment with a GRPO training pipeline, we show that agents can learn to reason over lattice structures and navigate complex tradeoffs to discover optimized scientific solutions. | |
| <div align="center"> | |
| <p>Built for the <b>Meta PyTorch OpenEnv Hackathon</b> by Arsh Pathan</p> | |
| </div> | |