ixim's picture
Release verified Turbo INT8 with eight-step paired evaluation
a5e8975 verified
|
Raw History Blame Contribute Delete
2.42 kB

Informal release evaluation: BF16 / INT8 comparison

This community evaluation is provided for reference only and does not represent any official evaluation.

18 paired outputs on NVIDIA GeForce RTX 5090; 1024×1024, 8 steps, offload=model, CFG=1, KV cache enabled.

Warmup excluded. Pixel metrics measure drift, not semantic quality. Sweater editing pairs use the same BF16 portrait input. The suite is small and does not establish a general quality ranking.

Case Seed BF16 s INT8 s BF16 peak GiB INT8 peak GiB RGB MAE (white)
portrait 42 13.70 8.14 16.40 10.02 0.0083
portrait 123 11.12 7.72 16.40 10.02 0.0363
english_text 42 10.88 7.87 16.41 10.03 0.0272
english_text 123 10.80 7.84 16.41 10.03 0.0393
chinese_text 42 11.05 7.79 16.41 10.03 0.0239
chinese_text 123 10.95 7.88 16.41 10.03 0.0582
composition 42 10.68 7.87 16.41 10.02 0.0120
composition 123 10.66 7.87 16.41 10.02 0.0090
texture 42 10.91 7.80 16.40 10.02 0.0340
texture 123 10.66 7.78 16.40 10.02 0.0309
rgba 42 10.80 7.93 16.40 10.01 0.0239
rgba 123 11.04 7.94 16.40 10.01 0.0246
edit 1000042 12.79 9.72 19.08 12.62 0.0038
edit 1000123 12.81 9.58 19.08 12.62 0.0034
young_chinese_woman 42 10.91 7.91 16.45 10.06 0.0109
young_chinese_woman 123 10.92 7.74 16.45 10.06 0.0095
edit_boat 20261009 12.93 9.46 19.09 12.62 0.0069
edit_boat 20261010 12.91 9.48 19.09 12.62 0.0067

Summary

Mean latency: BF16 11.47s; INT8 8.24s.

Interactive-sized side-by-side gallery. Raw data: comparison.csv, bf16/records.jsonl, int8/records.jsonl. Environment records include package versions and loading overhead.

Qualitative observations.

Interpretation limits

  • No CLIP, OCR, human preference, FID or benchmark leaderboard score is claimed.
  • Peak CUDA allocated/reserved memory excludes other processes and display usage.
  • Measured call latency includes transfers; disk writing and model loading are excluded.
  • Paired images may diverge with quantization even when both remain plausible.
  • Editing and transparency should be inspected in the gallery, including preserved details.