ixim's picture
Release verified Turbo INT8 with eight-step paired evaluation
a5e8975 verified
|
Raw History Blame Contribute Delete
2.42 kB
# Informal release evaluation: BF16 / INT8 comparison
This community evaluation is provided for reference only and does not represent any official evaluation.
18 paired outputs on NVIDIA GeForce RTX 5090; 1024×1024, 8 steps, offload=model, CFG=1, KV cache enabled.
Warmup excluded. Pixel metrics measure drift, not semantic quality. Sweater editing pairs use the same BF16 portrait input. The suite is small and does not establish a general quality ranking.
| Case | Seed | BF16 s | INT8 s | BF16 peak GiB | INT8 peak GiB | RGB MAE (white) |
|---|---:|---:|---:|---:|---:|---:|
| portrait | 42 | 13.70 | 8.14 | 16.40 | 10.02 | 0.0083 |
| portrait | 123 | 11.12 | 7.72 | 16.40 | 10.02 | 0.0363 |
| english_text | 42 | 10.88 | 7.87 | 16.41 | 10.03 | 0.0272 |
| english_text | 123 | 10.80 | 7.84 | 16.41 | 10.03 | 0.0393 |
| chinese_text | 42 | 11.05 | 7.79 | 16.41 | 10.03 | 0.0239 |
| chinese_text | 123 | 10.95 | 7.88 | 16.41 | 10.03 | 0.0582 |
| composition | 42 | 10.68 | 7.87 | 16.41 | 10.02 | 0.0120 |
| composition | 123 | 10.66 | 7.87 | 16.41 | 10.02 | 0.0090 |
| texture | 42 | 10.91 | 7.80 | 16.40 | 10.02 | 0.0340 |
| texture | 123 | 10.66 | 7.78 | 16.40 | 10.02 | 0.0309 |
| rgba | 42 | 10.80 | 7.93 | 16.40 | 10.01 | 0.0239 |
| rgba | 123 | 11.04 | 7.94 | 16.40 | 10.01 | 0.0246 |
| edit | 1000042 | 12.79 | 9.72 | 19.08 | 12.62 | 0.0038 |
| edit | 1000123 | 12.81 | 9.58 | 19.08 | 12.62 | 0.0034 |
| young_chinese_woman | 42 | 10.91 | 7.91 | 16.45 | 10.06 | 0.0109 |
| young_chinese_woman | 123 | 10.92 | 7.74 | 16.45 | 10.06 | 0.0095 |
| edit_boat | 20261009 | 12.93 | 9.46 | 19.09 | 12.62 | 0.0069 |
| edit_boat | 20261010 | 12.91 | 9.48 | 19.09 | 12.62 | 0.0067 |
## Summary
Mean latency: BF16 11.47s; INT8 8.24s.
[Interactive-sized side-by-side gallery](comparison.html). Raw data: comparison.csv, bf16/records.jsonl, int8/records.jsonl. Environment records include package versions and loading overhead.
[Qualitative observations](qualitative.md).
## Interpretation limits
- No CLIP, OCR, human preference, FID or benchmark leaderboard score is claimed.
- Peak CUDA allocated/reserved memory excludes other processes and display usage.
- Measured call latency includes transfers; disk writing and model loading are excluded.
- Paired images may diverge with quantization even when both remain plausible.
- Editing and transparency should be inspected in the gallery, including preserved details.