| --- |
| license: cc-by-nc-sa-4.0 |
| library_name: pytorch |
| tags: |
| - image-fusion |
| - infrared-visible |
| - perceptual-quality |
| - pairwise-preference |
| - bradley-terry |
| pipeline_tag: image-classification |
| --- |
| |
| # LPIFM — Learned Perceptual Image Fusion Measure |
|
|
| Pairwise perceptual preference model for **infrared–visible image fusion (IVIF)** ranking. |
|
|
| Given an IR source, a VI source, and two fused candidates, LPIFM predicts **A better**, **B better**, or **Tie**. Pairwise decisions can be aggregated with tie-aware Bradley–Terry (T-BT) to rank a method pool. |
|
|
| Code and examples: [github.com/HaoranLiu507/LPIFM](https://github.com/HaoranLiu507/LPIFM) |
|
|
| ## Files |
|
|
| | File | Role | |
| | --- | --- | |
| | `lpifm_vifb_baseline_v1.pt` | Main VIFB-trained public checkpoint | |
| | `inference_config.yaml` | Decode defaults (`t`, `T_cal`, image size) | |
|
|
| EVAFusion fine-tuned weights are **not** hosted here; see the Zenodo companion archive when published. |
|
|
| **Note on checkpoint size.** |
| `lpifm_vifb_baseline_v1.pt` is about **2 GB** because it is a full training checkpoint: three weight copies (`swa` / `ema` / `model`, about 0.4 GB each) plus the optimizer (about 0.8 GB). To **use LPIFM** only, keep `swa_state_dict` (+ `config`); about 0.4 GB is enough. Inference already loads SWA by default. |
|
|
| ## Architecture |
|
|
| - Backbone: **ConvNeXt-V2** (`convnextv2_base.fcmae_ft_in22k_in1k_384` via `timm`) |
| - Input size: **384 × 384** |
| - Task: source-conditioned pairwise preference scoring (ternary A / B / Tie) |
|
|
| ## Decode / inference defaults |
|
|
| | Parameter | Value | |
| | --- | --- | |
| | Tie threshold `t` | `0.3` | |
| | Calibration temperature `T_cal` | `1.0` | |
| | Image size | `384` | |
|
|
| ```text |
| d_cal = d / T_cal |
| d_cal > 0.3 → A better (0) |
| d_cal < -0.3 → B better (1) |
| otherwise → Tie (2) |
| ``` |
|
|
| ## Quick start |
|
|
| ```bash |
| # from the GitHub repository |
| python scripts/download_assets.py --source hf |
| python predict.py \ |
| --config configs/release_inference.yaml \ |
| --dataset_root Dataset/VIFB \ |
| --ckpt checkpoints/lpifm_vifb_baseline_v1.pt \ |
| --image_name carLight.jpg \ |
| --a_dir U2Fusion \ |
| --b_dir SeAFusion |
| ``` |
|
|
| Or download this file directly: |
|
|
| ```bash |
| huggingface-cli download FengShaner/LPIFM lpifm_vifb_baseline_v1.pt \ |
| --local-dir checkpoints |
| ``` |
|
|
| ## Intended use |
|
|
| - Pairwise **perceptual preference** for **IVIF method ranking** |
| - Research / offline evaluation under the LPIFM protocol |
| - Not a general-purpose IQA model for arbitrary natural-image aesthetics |
|
|
| ## Limitations |
|
|
| - Trained and validated under a specific preference-collection and ranking protocol |
| - Protocol mismatch (new dataset, different rater instructions, different method pools) may reduce agreement; fine-tuning may be required |
| - Non-commercial weights license (see below) |
|
|
| ## Licenses |
|
|
| | Artifact | License | |
| | --- | --- | |
| | Model weights on this Hub repo | [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) | |
| | Source code on GitHub | [AGPL-3.0](https://github.com/HaoranLiu507/LPIFM/blob/main/LICENSE) | |
|
|
| Weights do **not** inherit AGPL; code does **not** inherit CC BY-NC-SA. |
|
|
| ## Links |
|
|
| - GitHub: https://github.com/HaoranLiu507/LPIFM |
| - Hugging Face: https://huggingface.co/FengShaner/LPIFM |
| - Zenodo archival DOI: to be added after deposit publication |
|
|
| ## Citation |
|
|
| See [`CITATION.cff`](https://github.com/HaoranLiu507/LPIFM/blob/main/CITATION.cff) in the GitHub repository. |
|
|
