File size: 2,492 Bytes
46786cb 5adf028 46786cb 5adf028 bcd0911 5adf028 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
license: llama2
base_model:
- lmsys/vicuna-7b-v1.5
tags:
- vision-language-model
- TEM,microscopy
- materials-science
- llava
- scientific-VLM
language:
- en
pipeline_tag: image-text-to-text
---
# ATOMIC-LLaVA
ATOMIC-LLaVA is a domain-specific Vision-Language Model for Transmission Electron Microscopy (TEM), fine-tuned from LLaVA-v1.5-7B (Vicuna-v1.5-7B) using a two-stage training pipeline on 32,564 TEM subfigures collected from Nature portfolio journals.
This model is introduced in the ECCV 2026 paper:
> **ATOMIC: A Domain-Specific Vision-Language Model for Transmission Electron Microscopy**
For code, evaluation scripts, and dataset, please refer to our GitHub repository:
👉 [https://github.com/SemiMIRTLab/ATOMIC](https://github.com/SemiMIRTLab/ATOMIC)
---
## Model Details
| | |
|---|---|
| **Base Model** | LLaVA-v1.5-7B (Vicuna-v1.5-7B) |
| **Training Stage** | Stage 1 (alignment) + Stage 2 (instruction tuning) |
| **Training Data** | 120K Stage 1 pairs + 60K Stage 2 conversations |
| **Domain** | Transmission Electron Microscopy (TEM) |
| **Modalities** | CTEM, HR-TEM, STEM, Diffraction |
---
## Important: Inference Requirements
ATOMIC-LLaVA is built on LLaVA and **cannot be loaded directly via `transformers`**. Inference requires the LLaVA repository.
**Step 1 — Clone LLaVA:**
```bash
git clone https://github.com/haotian-liu/LLaVA.git
cd LLaVA
pip install -e .
```
**Step 2 — Download weights:**
```python
from huggingface_hub import snapshot_download
snapshot_download(repo_id="LabSmart/ATOMIC-LLaVA", local_dir="./ATOMIC-LLaVA")
```
**Step 3 — Run inference using our evaluation scripts:**
Please refer to `evaluation/` in our GitHub repository for inference and evaluation scripts.
---
## Training Data
Training data is available on HuggingFace:
👉 [https://huggingface.co/datasets/LabSmart/ATOMIC_dataset](https://huggingface.co/datasets/LabSmart/ATOMIC_dataset)
---
## Citation
```bibtex
@inproceedings{atomic2026eccv,
title = {ATOMIC: A Domain-Specific Vision-Language Model
for Transmission Electron Microscopy},
author = {Tu, C. and Hsu, Shu-han and others},
booktitle = {Proceedings of ECCV 2026},
year = {2026},
note = {BibTeX will be updated upon publication}
}
```
---
## License
This model is released under the [LLaMA 2 Community License](https://ai.meta.com/llama/license/). It is intended for academic research purposes only and may not be used for commercial purposes. |