---
license: apache-2.0
base_model: unsloth/mistral-7b-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- materials
- raman
- spectroscopy
- inverse-design
- crystal-structure
- lora
- jarvis
---

# RamanGPT (Mistral-7B): Raman spectrum to crystal structure

LoRA adapter fine-tuned from `unsloth/mistral-7b-bnb-4bit` on
[`knc6/ramangpt_dft`](https://huggingface.co/datasets/knc6/ramangpt_dft). This is
the **inverse** model of the RamanGPT framework: given a chemical formula and the
main Raman peaks, it generates a crystal structure (lattice + fractional
coordinates). The **forward** model (ALIGNN) predicts the 200-bin Raman spectrum
(50--1000 cm$^{-1}$) from a structure, and an inverse->relax->forward loop couples
the two. Part of the AtomGPT spectroscopy suite (cf. IRGPT, DiffractGPT).

## Training
QLoRA (initialized from the DiffractGPT explicit-formula checkpoint), LoRA r=16,
alpha=16 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs,
4-bit base, seed 3407.

## Evaluation
- **Inverse** (508 held-out materials): reduced-formula consistency **90.7%**,
  lattice-parameter MAE **1.16--2.36 Angstrom**, versus **17.2%** formula
  consistency for a formula-conditioned retrieval baseline given identical inputs.
- **Forward** (ALIGNN, 5,099-material Computational Raman Database): **43.4%** of
  held-out materials reach an active-region cosine similarity >= 0.354, capturing
  the qualitative mode structure; it also agrees qualitatively with an
  experimental 1T-VSe2 spectrum absent from training.

## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "unsloth/mistral-7b-bnb-4bit"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, device_map="auto"),
    "knc6/ramangpt_dft_model")
prompt = ("### Instruction:\nBelow is a description of a material.\n### Input:\n"
          "The chemical formula is: MoS2.\nThe Raman spectrum shows main peaks at: "
          "383 cm-1(1.0), 408 cm-1(0.9).\nGenerate atomic structure description with "
          "lattice lengths, angles, coordinates and atom types.\n### Output:\n")
ids = tok(prompt, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:],
                 skip_special_tokens=True))
```

## Links
- Dataset: https://huggingface.co/datasets/knc6/ramangpt_dft
- Code (forward + inverse, any base LLM): https://github.com/atomgptlab/ramangpt
- Web app: https://atomgpt.org/raman ; Paper: arXiv:2606.03764