Spaces:
Sleeping
title: MaterialForge
emoji: 🔬
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: mit
base_path: /web
MaterialForge
Inverse Autonomous Synthesis of Crystalline Atomic Lattices
MaterialForge frames materials discovery as an inverse design problem. Given target physical properties, the agent must discover a lattice arrangement that produces them. The environment is built to train and evaluate LLM agents on a scientific discovery workflow rather than a static prompt task.
Live Links
| Feature | URL |
|---|---|
| Environment API | https://huggingface.co/spaces/ArshPathan/material_forge_env |
| Interactive Dashboard | https://huggingface.co/spaces/ArshPathan/material_forge_env/playground |
| Training Blog | BLOG.md |
| Research / Strategy Notes | RESEARCH.md |
| Training Notebook | training/MaterialForge_GRPO_Training.ipynb |
The Inverse Design Goal
The core task is:
Given target values for hardness, conductivity, thermal resistance, and elasticity, the agent must discover a lattice arrangement that produces the target properties.
The agent does not solve this in one shot. It must:
- observe the current 8x8 lattice state
- discover how the placement of different atom species influences global properties
- see how actions change material properties in real-time
- recover from structural instability or budget overruns
- build toward a high-quality crystalline phase that achieves the target performance
This makes MaterialForge a prime example of:
- Theme #3.1: World Modeling
- Theme #2: Long-Horizon Planning
Why This Matters
MaterialForge is inspired by the complexity of inverse material design. Even though the simulation is simplified to an 8x8 lattice, the environment captures critical scientific principles:
- Percolation pathways for conductivity
- Coordination-driven stability for local structure quality
- Phase / order formation for crystalline vs amorphous behavior
- Budget-constrained design where structure discovery must be both accurate and efficient
Training Approach
We used a GRPO-based notebook pipeline built with:
- OpenEnv for environment structure
- TRL for RL post-training
- Unsloth for efficient 4-bit QLoRA finetuning
- Qwen/Qwen3-0.6B as the base model for tool-using policy training
For the Grand Finale Submission, we scaled the training using a high-memory server with ~42GB VRAM, enabling 8 generations per prompt and real-time gradient updates for maximum stability during the discovery process.
Training Evidence
The training results demonstrate measurable agent improvement in discovering high-performing lattice structures, achieving a peak episodic reward of 0.86+:
High-Performance Setup
- Base Model:
Qwen/Qwen3-0.6B(Unsloth 4-bit) - Generations per Prompt:
8(leveraging high-memory hardware for stable advantage estimation) - Max Completion Length:
512(allowing for deep scientific reasoning) - Hardware: High-memory server (~42GB VRAM)
Reward Curve
The reward curve shows the agent successfully learning the structural tradeoffs required to reach target properties.
Loss Curve
The loss plot confirms stable policy optimization over the training steps.
Baseline Comparison
The trained agent significantly outperforms a random baseline, demonstrating a learned ability to discover valid, high-property lattice arrangements.
Final Summary
MaterialForge demonstrates how reinforcement learning can be applied to inverse scientific design problems, where an agent must search the combinatorial space of structures that produce desired physical properties. By combining an OpenEnv environment with a GRPO training pipeline, we show that agents can learn to reason over lattice structures and navigate complex tradeoffs to discover optimized scientific solutions.
Built for the Meta PyTorch OpenEnv Hackathon by Arsh Pathan