Hugging Face's logo

knc6
/
ramangpt_dft_model

Text Generation
PEFT
Safetensors
materials
raman
spectroscopy
inverse-design
crystal-structure
lora
jarvis
Model card Files
xet
 Community
ramangpt_dft_model / README.md
knc6's picture
knc6
Upload README.md with huggingface_hub
e31d6e5 verified
preview code
|
Raw History Blame Contribute Delete
2.62 kB
metadata 
license: apache-2.0
base_model: unsloth/mistral-7b-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
  - materials
  - raman
  - spectroscopy
  - inverse-design
  - crystal-structure
  - lora
  - jarvis

RamanGPT (Mistral-7B): Raman spectrum to crystal structure

LoRA adapter fine-tuned from unsloth/mistral-7b-bnb-4bit on knc6/ramangpt_dft. This is the inverse model of the RamanGPT framework: given a chemical formula and the main Raman peaks, it generates a crystal structure (lattice + fractional coordinates). The forward model (ALIGNN) predicts the 200-bin Raman spectrum (50--1000 cm$^{-1}$) from a structure, and an inverse->relax->forward loop couples the two. Part of the AtomGPT spectroscopy suite (cf. IRGPT, DiffractGPT).

Training

QLoRA (initialized from the DiffractGPT explicit-formula checkpoint), LoRA r=16, alpha=16 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs, 4-bit base, seed 3407.

Evaluation

  • Inverse (508 held-out materials): reduced-formula consistency 90.7%, lattice-parameter MAE 1.16--2.36 Angstrom, versus 17.2% formula consistency for a formula-conditioned retrieval baseline given identical inputs.
  • Forward (ALIGNN, 5,099-material Computational Raman Database): 43.4% of held-out materials reach an active-region cosine similarity >= 0.354, capturing the qualitative mode structure; it also agrees qualitatively with an experimental 1T-VSe2 spectrum absent from training.

Usage 

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "unsloth/mistral-7b-bnb-4bit"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, device_map="auto"),
    "knc6/ramangpt_dft_model")
prompt = ("### Instruction:\nBelow is a description of a material.\n### Input:\n"
          "The chemical formula is: MoS2.\nThe Raman spectrum shows main peaks at: "
          "383 cm-1(1.0), 408 cm-1(0.9).\nGenerate atomic structure description with "
          "lattice lengths, angles, coordinates and atom types.\n### Output:\n")
ids = tok(prompt, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:],
                 skip_special_tokens=True))

Links