| --- |
| license: apache-2.0 |
| base_model: unsloth/mistral-7b-bnb-4bit |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: |
| - materials |
| - raman |
| - spectroscopy |
| - inverse-design |
| - crystal-structure |
| - lora |
| - jarvis |
| --- |
| |
| # RamanGPT (Mistral-7B): Raman spectrum to crystal structure |
|
|
| LoRA adapter fine-tuned from `unsloth/mistral-7b-bnb-4bit` on |
| [`knc6/ramangpt_dft`](https://huggingface.co/datasets/knc6/ramangpt_dft). This is |
| the **inverse** model of the RamanGPT framework: given a chemical formula and the |
| main Raman peaks, it generates a crystal structure (lattice + fractional |
| coordinates). The **forward** model (ALIGNN) predicts the 200-bin Raman spectrum |
| (50--1000 cm$^{-1}$) from a structure, and an inverse->relax->forward loop couples |
| the two. Part of the AtomGPT spectroscopy suite (cf. IRGPT, DiffractGPT). |
|
|
| ## Training |
| QLoRA (initialized from the DiffractGPT explicit-formula checkpoint), LoRA r=16, |
| alpha=16 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs, |
| 4-bit base, seed 3407. |
|
|
| ## Evaluation |
| - **Inverse** (508 held-out materials): reduced-formula consistency **90.7%**, |
| lattice-parameter MAE **1.16--2.36 Angstrom**, versus **17.2%** formula |
| consistency for a formula-conditioned retrieval baseline given identical inputs. |
| - **Forward** (ALIGNN, 5,099-material Computational Raman Database): **43.4%** of |
| held-out materials reach an active-region cosine similarity >= 0.354, capturing |
| the qualitative mode structure; it also agrees qualitatively with an |
| experimental 1T-VSe2 spectrum absent from training. |
|
|
| ## Usage |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| import torch |
| base = "unsloth/mistral-7b-bnb-4bit" |
| tok = AutoTokenizer.from_pretrained(base) |
| model = PeftModel.from_pretrained( |
| AutoModelForCausalLM.from_pretrained(base, device_map="auto"), |
| "knc6/ramangpt_dft_model") |
| prompt = ("### Instruction:\nBelow is a description of a material.\n### Input:\n" |
| "The chemical formula is: MoS2.\nThe Raman spectrum shows main peaks at: " |
| "383 cm-1(1.0), 408 cm-1(0.9).\nGenerate atomic structure description with " |
| "lattice lengths, angles, coordinates and atom types.\n### Output:\n") |
| ids = tok(prompt, return_tensors="pt").to(model.device) |
| print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:], |
| skip_special_tokens=True)) |
| ``` |
|
|
| ## Links |
| - Dataset: https://huggingface.co/datasets/knc6/ramangpt_dft |
| - Code (forward + inverse, any base LLM): https://github.com/atomgptlab/ramangpt |
| - Web app: https://atomgpt.org/raman ; Paper: arXiv:2606.03764 |
| |
