| # Informal release evaluation: BF16 / INT8 comparison |
|
|
| This community evaluation is provided for reference only and does not represent any official evaluation. |
|
|
| 4 paired outputs on NVIDIA GeForce RTX 5090; 2048×2048, 8 steps, offload=model, CFG=1, KV cache enabled. |
|
|
| Warmup excluded. Pixel metrics measure drift, not semantic quality. The suite is small and does not establish a general quality ranking. |
|
|
| | Case | Seed | BF16 s | INT8 s | BF16 peak GiB | INT8 peak GiB | RGB MAE (white) | |
| |---|---:|---:|---:|---:|---:|---:| |
| | portrait | 42 | 45.85 | 33.40 | 26.89 | 26.89 | 0.0100 | |
| | portrait | 123 | 46.31 | 32.99 | 26.89 | 26.89 | 0.0169 | |
| | edit_boat | 20261009 | 87.84 | 52.01 | 26.93 | 26.93 | 0.0090 | |
| | edit_boat | 20261010 | 65.72 | 56.79 | 26.93 | 26.93 | 0.0071 | |
|
|
| ## Summary |
|
|
| Mean latency: BF16 61.43s; INT8 43.79s. |
|
|
| [Interactive-sized side-by-side gallery](comparison.html). Raw data: comparison.csv, bf16/records.jsonl, int8/records.jsonl. Environment records include package versions and loading overhead. |
|
|
| [Qualitative observations](qualitative.md). |
|
|
| ## Interpretation limits |
|
|
| - No CLIP, OCR, human preference, FID or benchmark leaderboard score is claimed. |
| - Peak CUDA allocated/reserved memory excludes other processes and display usage. |
| - Measured call latency includes transfers; disk writing and model loading are excluded. |
| - Paired images may diverge with quantization even when both remain plausible. |
| - Editing and transparency should be inspected in the gallery, including preserved details. |
|
|
