ThreeSixNine/florence2-estate-antiques

Evaluate on your photosTrain a second epoch

Florence-2-large LoRA — estate antique identifier

LoRA adapter for florence-community/Florence-2-large trained to identify antiques from estate-sale-style photos and emit structured JSON: {"item", "era", "maker", "reasoning", "confidence"}. Deployed as an MCP-enabled Gradio Space: ThreeSixNine/estate-antique-identifier.

Training data

ThreeSixNine/estate-antiques-merged — 6,000 train rows, one unified schema (image, caption, category, confidence, source, task, prompt, target, raw_name_zh):

• 3,500 naming rows from SII-Monument-Valley/CiQi-VQA (ECCV 2026, [2603.28474]): canonical porcelain names parsed from the connoisseurship essays (98% extraction rate), translated to English with a domain dictionary (dynasty/kiln/glaze/shape/motif). Confidence 0.55–0.7.
• 2,500 captioning rows from the RedCaps antiques_2020 slice (kdexd/red_caps, [2111.11431]): community captions, ~95% image-URL survival measured before use. Confidence 0.5.

No price data exists in either source, so "estimated resale range" is omitted — the Space returns null for that field. Value estimation needs real sold-price labels (a separate data-collection effort; marketplaces were deliberately not scraped in this run).

Eval split: 386 multiple-choice rows from CiQi-Bench (test.jsonl), translated to English, held out from training.

Training

• Base: florence-community/Florence-2-large (native transformers 5.22.2 class, fp32 weights + bf16 autocast)
• LoRA r=16, α=32, dropout 0.05, targets q_proj/k_proj/v_proj/out_proj (language model only), 4.72M trainable params (0.60%)
• HF Trainer, 1 epoch over 6,000 rows (750 steps, batch 8), lr 1e-4 cosine, warmup 30 steps, gradient checkpointing, a10g-small
• Final avg train loss 0.94 (final step 0.66); all 288 adapter tensors changed; adapter save/reload verified

Note on a discarded first run: an initial training pass accidentally included the target text in the encoder input; the model learned to copy the answer (train loss 0.0016, zero generalization). That adapter was replaced by this one; the collator now hard-asserts no answer text in the encoder input.

Results

Evaluated on the same 386-row CiQi-Bench subset, deterministic per-option log-prob scoring (same method before and after):

Metric	Zero-shot baseline	Post-train
MC accuracy, per-option log-prob	18.65%	19.69% (+1.04 pts)
MC accuracy, JSON-format continuation	20.98%	20.98% (parity)
Free-form generation: item token-F1 ≥ 0.5 vs ground truth	~0 (baseline echoes the prompt template)	56% (n=200)
Free-form generation: exact match	0%	0%

Honest reading: one epoch over 6k rows teaches the model the format and vocabulary (it now emits real kiln/era/shape descriptions where the base model emits prompt echo), and lifts the primary MC metric by ~1 point, but does not approach the benchmark's strong baselines (GPT-5: 75.8% MC on the full CiQi-Bench; CiQi-Agent-7B: 81.5%). MC accuracy remains below the 25% chance line for all four options — fine-grained porcelain attribution from a 770M model after one light epoch is genuinely hard; more epochs, more data, or a larger backbone are the obvious next levers.

Usage

from peft import PeftModel
from transformers import AutoProcessor, Florence2ForConditionalGeneration

processor = AutoProcessor.from_pretrained("florence-community/Florence-2-large")
model = Florence2ForConditionalGeneration.from_pretrained("florence-community/Florence-2-large")
model = PeftModel.from_pretrained(model, "ThreeSixNine/florence2-estate-antiques", subfolder="adapter")
inputs = processor(text="Identify this porcelain item. Reply with JSON: item, era, maker, reasoning.",
                   images=[pil_image], return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=192, num_beams=3)
print(processor.tokenizer.batch_decode(out, skip_special_tokens=True)[0])

Limitations

• Chinese porcelain only for naming (CiQi heritage); RedCaps rows add general antiques captioning, not attribution.
• confidence values are heuristic labels from data prep, not calibrated probabilities.
• English names are dictionary translations of Chinese canonical names; some segments unmatched (flagged with lower confidence in the dataset).
• MC eval subset is the overall (naming) type only; per-type breakdowns are not available in this run's eval split.
• License: CC-BY-NC-4.0 (inherited from CiQi-VQA) — non-commercial use.

Reproduction

Training and eval scripts, plus the live Trackio dashboard (ThreeSixNine/florence2-estate-antiques-trackio), cover: zero-shot baseline, 50-step smoke tests (weights-verified), and both finetune runs. MCP tool: identify_antique(image_url) -> JSON on the Space.