# AutoencoderKLCosmos [Cosmos Tokenizers](https://github.com/NVIDIA/Cosmos-Tokenizer). Model yang didukung: - [nvidia/Cosmos-1.0-Tokenizer-CV8x8x8](https://huggingface.co/nvidia/Cosmos-1.0-Tokenizer-CV8x8x8) Model dapat dimuat dengan potongan kode berikut. ```python from diffusers import AutoencoderKLCosmos vae = AutoencoderKLCosmos.from_pretrained("nvidia/Cosmos-1.0-Tokenizer-CV8x8x8", subfolder="vae") ``` ## AutoencoderKLCosmos[[diffusers.AutoencoderKLCosmos]] #### diffusers.AutoencoderKLCosmos[[diffusers.AutoencoderKLCosmos]] ```python diffusers.AutoencoderKLCosmos(in_channels: int = 3, out_channels: int = 3, latent_channels: int = 16, encoder_block_out_channels: tuple[int, ...] = (128, 256, 512, 512), decode_block_out_channels: tuple[int, ...] = (256, 512, 512, 512), attention_resolutions: tuple[int, ...] = (32,), resolution: int = 1024, num_layers: int = 2, patch_size: int = 4, patch_type: str = 'haar', scaling_factor: float = 1.0, spatial_compression_ratio: int = 8, temporal_compression_ratio: int = 8, latents_mean: list[float] | None = [0.11362758, -0.0171717, 0.03071163, 0.02046862, 0.01931456, 0.02138567, 0.01999342, 0.02189187, 0.02011935, 0.01872694, 0.02168613, 0.02207148, 0.01986941, 0.01770413, 0.02067643, 0.02028245, 0.19125476, 0.04556972, 0.0595558, 0.05315534, 0.05496629, 0.05356264, 0.04856596, 0.05327453, 0.05410472, 0.05597149, 0.05524866, 0.05181874, 0.05071663, 0.05204537, 0.0564108, 0.05518042, 0.01306714, 0.03341161, 0.03847246, 0.02810185, 0.02790166, 0.02920026, 0.02823597, 0.02631033, 0.0278531, 0.02880507, 0.02977769, 0.03145441, 0.02888389, 0.03280773, 0.03484927, 0.03049198, -0.00197727, 0.07534957, 0.04963879, 0.05530893, 0.05410828, 0.05252541, 0.05029899, 0.05321025, 0.05149245, 0.0511921, 0.04643495, 0.04604527, 0.04631618, 0.04404101, 0.04403536, 0.04499495, -0.02994183, -0.04787003, -0.01064558, -0.01779824, -0.01490502, -0.02157517, -0.0204778, -0.02180816, -0.01945375, -0.02062863, -0.02192209, -0.02520639, -0.02246656, -0.02427533, -0.02683363, -0.02762006, 0.08019473, -0.13005368, -0.07568636, -0.06082374, -0.06036175, -0.05875364, -0.05921887, -0.05869788, -0.05273941, -0.052565, -0.05346428, -0.05456541, -0.053657, -0.05656897, -0.05728589, -0.05321847, 0.16718403, -0.00390146, 0.0379406, 0.0356561, 0.03554131, 0.03924074, 0.03873615, 0.04187329, 0.04226924, 0.04378717, 0.04684274, 0.05117614, 0.04547792, 0.05251586, 0.05048339, 0.04950784, 0.09564418, 0.0547128, 0.08183969, 0.07978633, 0.08076023, 0.08108605, 0.08011818, 0.07965573, 0.08187773, 0.08350263, 0.08101469, 0.0786941, 0.0774442, 0.07724521, 0.07830418, 0.07599796, -0.04987567, 0.05923908, -0.01058746, -0.01177603, -0.01116162, -0.01364149, -0.01546014, -0.0117213, -0.01780043, -0.01648314, -0.02100247, -0.02104417, -0.02482123, -0.02611689, -0.02561143, -0.02597336, -0.05364667, 0.08211684, 0.04686937, 0.04605641, 0.04304186, 0.0397355, 0.03686767, 0.04087112, 0.03704741, 0.03706401, 0.03120073, 0.03349091, 0.03319963, 0.03205781, 0.03195127, 0.03180481, 0.16427967, -0.11048453, -0.04595276, -0.04982893, -0.05213465, -0.04809378, -0.05080318, -0.04992863, -0.04493337, -0.0467619, -0.04884703, -0.04627892, -0.04913311, -0.04955709, -0.04533982, -0.04570218, -0.10612928, -0.05121198, -0.06761009, -0.07251801, -0.07265285, -0.07417855, -0.07202412, -0.07499027, -0.07625481, -0.07535747, -0.07638787, -0.07920305, -0.07596069, -0.07959418, -0.08265036, -0.07955471, -0.16888915, 0.0753242, 0.04062594, 0.03375093, 0.03337452, 0.03699376, 0.03651138, 0.03611023, 0.03555622, 0.03378554, 0.0300498, 0.03395559, 0.02941847, 0.03156432, 0.03431173, 0.03016853, -0.03415358, -0.01699573, -0.04029295, -0.04912157, -0.0498858, -0.04917918, -0.04918056, -0.0525189, -0.05325506, -0.05341973, -0.04983329, -0.04883146, -0.04985548, -0.04736718, -0.0462027, -0.04836091, 0.02055675, 0.03419799, -0.02907669, -0.04350509, -0.04156144, -0.04234421, -0.04446109, -0.04461774, -0.04882839, -0.04822346, -0.04502493, -0.0506244, -0.05146913, -0.04655267, -0.04862994, -0.04841615, 0.20312774, -0.07208502, -0.03635615, -0.03556088, -0.04246174, -0.04195838, -0.04293778, -0.04071276, -0.04240569, -0.04125213, -0.04395144, -0.03959096, -0.04044993, -0.04015875, -0.04088107, -0.03885176], latents_std: list[float] | None = [0.56700271, 0.65488982, 0.65589428, 0.66524369, 0.66619784, 0.6666382, 0.6720838, 0.66955978, 0.66928875, 0.67108786, 0.67092526, 0.67397463, 0.67894882, 0.67668313, 0.67769569, 0.67479557, 0.85245121, 0.8688373, 0.87348086, 0.88459337, 0.89135885, 0.8910504, 0.89714909, 0.89947474, 0.90201765, 0.90411824, 0.90692616, 0.90847772, 0.90648711, 0.91006982, 0.91033435, 0.90541548, 0.84960359, 0.85863352, 0.86895317, 0.88460612, 0.89245003, 0.89451706, 0.89931005, 0.90647358, 0.90338236, 0.90510076, 0.91008312, 0.90961218, 0.9123717, 0.91313171, 0.91435546, 0.91565102, 0.91877103, 0.85155135, 0.857804, 0.86998034, 0.87365264, 0.88161767, 0.88151032, 0.88758916, 0.89015514, 0.89245576, 0.89276224, 0.89450496, 0.90054202, 0.89994133, 0.90136105, 0.90114892, 0.77755755, 0.81456852, 0.81911844, 0.83137071, 0.83820474, 0.83890373, 0.84401101, 0.84425181, 0.84739357, 0.84798753, 0.85249585, 0.85114998, 0.85160935, 0.85626358, 0.85677862, 0.85641026, 0.69903517, 0.71697885, 0.71696913, 0.72583169, 0.72931731, 0.73254126, 0.73586977, 0.73734969, 0.73664582, 0.74084908, 0.74399322, 0.74471819, 0.74493188, 0.74824578, 0.75024873, 0.75274801, 0.8187142, 0.82251883, 0.82616025, 0.83164483, 0.84072375, 0.8396467, 0.84143305, 0.84880769, 0.8503468, 0.85196948, 0.85211051, 0.85386664, 0.85410017, 0.85439342, 0.85847849, 0.85385275, 0.67583984, 0.68259847, 0.69198853, 0.69928843, 0.70194328, 0.70467001, 0.70755547, 0.70917857, 0.71007699, 0.70963502, 0.71064079, 0.71027333, 0.71291167, 0.71537536, 0.71902508, 0.71604162, 0.72450989, 0.71979928, 0.72057378, 0.73035461, 0.73329622, 0.73660028, 0.73891461, 0.74279994, 0.74105692, 0.74002433, 0.74257588, 0.74416119, 0.74543899, 0.74694443, 0.74747062, 0.74586403, 0.90176988, 0.90990674, 0.91106802, 0.92163783, 0.92390233, 0.93056196, 0.93482202, 0.93642414, 0.93858379, 0.94064975, 0.94078934, 0.94325715, 0.94955301, 0.94814706, 0.95144123, 0.94923073, 0.49853548, 0.64968109, 0.6427654, 0.64966393, 0.6487664, 0.65203559, 0.6584242, 0.65351611, 0.65464371, 0.6574859, 0.65626335, 0.66123748, 0.66121179, 0.66077942, 0.66040152, 0.66474909, 0.61986589, 0.69138134, 0.6884557, 0.6955843, 0.69765401, 0.70015347, 0.70529598, 0.70468754, 0.70399523, 0.70479989, 0.70887572, 0.71126866, 0.7097227, 0.71249932, 0.71231949, 0.71175605, 0.35586974, 0.68723857, 0.68973219, 0.69958478, 0.6943453, 0.6995818, 0.70980215, 0.69899458, 0.70271689, 0.70095056, 0.69912851, 0.70522696, 0.70392174, 0.70916915, 0.70585734, 0.70373541, 0.98101336, 0.89024764, 0.89607251, 0.90678179, 0.91308665, 0.91812348, 0.91980827, 0.92480654, 0.92635667, 0.92887944, 0.93338072, 0.93468094, 0.93619436, 0.93906063, 0.94191772, 0.94471723, 0.83202779, 0.84106231, 0.84463632, 0.85829508, 0.86319661, 0.86751342, 0.86914337, 0.87085921, 0.87286359, 0.87537396, 0.87931138, 0.88054478, 0.8811838, 0.88872558, 0.88942474, 0.88934827, 0.44025335, 0.63061613, 0.63110614, 0.63601959, 0.6395812, 0.64104342, 0.65019929, 0.6502797, 0.64355946, 0.64657205, 0.64847094, 0.64728117, 0.64972943, 0.65162975, 0.65328044, 0.64914775]) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl_cosmos.py#L879) **Parameter:** in_channels (`int`, defaultnya `3`) : Jumlah saluran input. out_channels (`int`, defaultnya `3`) : Jumlah saluran output. latent_channels (`int`, defaultnya `16`) : Jumlah kanal laten. encoder_block_out_channels (`tuple[int, ...]`, defaultnya `(128, 256, 512, 512)`) : Jumlah kanal output untuk setiap blok down encoder. decode_block_out_channels (`tuple[int, ...]`, defaultnya `(256, 512, 512, 512)`) : Jumlah kanal output untuk setiap blok up decoder. attention_resolutions (`tuple[int, ...]`, defaultnya `(32,)`) : daftar resolusi gambar/video di mana perhatian diterapkan. resolution (`int`, defaultnya `1024`) : Resolusi dasar gambar/video yang digunakan untuk menghitung apakah sebuah blok harus memiliki lapisan perhatian. num_layers (`int`, defaultnya `2`) : Jumlah blok resnet di setiap blok encoder/decoder. patch_size (`int`, defaultnya `4`) : Ukuran patch yang digunakan untuk mem-patch gambar/video input. patch_type (`str`, defaultnya `haar`) : Jenis patch yang digunakan untuk mem-patch gambar/video input. Bisa berupa `haar` atau `rearrange`. scaling_factor (`float`, defaultnya `1.0`) : Deviasi standar per komponen dari ruang laten yang dilatih, dihitung menggunakan batch pertama dari set pelatihan. Ini digunakan untuk menskalakan ruang laten agar memiliki varians satuan saat melatih model difusi. Laten diskalakan dengan rumus `z = z * scaling_factor` sebelum diteruskan ke model difusi. Saat decoding, laten diskalakan kembali ke skala aslinya dengan rumus: `z = 1 / scaling_factor * z`. Untuk detail lebih lanjut, lihat bagian 4.3.2 dan D.1 dari paper [High-Resolution Image Synthesis with Latent Diffusion Models](https://huggingface.co/papers/2112.10752). Tidak berlaku di Cosmos, tetapi kami menggunakan default 1.0 untuk konsistensi. spatial_compression_ratio (`int`, defaultnya `8`) : Rasio kompresi spasial yang diterapkan pada VAE. Jumlah blok downsample ditentukan menggunakan nilai ini. temporal_compression_ratio (`int`, defaultnya `8`) : Rasio kompresi temporal yang diterapkan pada VAE. Jumlah blok downsample ditentukan menggunakan nilai ini. Autoencoder yang digunakan dalam [Cosmos](https://huggingface.co/papers/2501.03575). #### decode[[diffusers.AutoencoderKLCosmos.decode]] ```python decode(z: torch.Tensor, return_dict: bool = True) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl_cosmos.py#L1062) #### encode[[diffusers.AutoencoderKLCosmos.encode]] ```python encode(x: torch.Tensor, return_dict: bool = True) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl_cosmos.py#L1040) #### enable_tiling[[diffusers.AutoencoderKLCosmos.enable_tiling]] ```python enable_tiling(tile_sample_min_height: int | None = None, tile_sample_min_width: int | None = None, tile_sample_min_num_frames: int | None = None, tile_sample_stride_height: float | None = None, tile_sample_stride_width: float | None = None, tile_sample_stride_num_frames: float | None = None) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl_cosmos.py#L1001) **Parameter:** tile_sample_min_height (`int`, *opsional*) : Tinggi minimum yang diperlukan agar sampel dipisahkan menjadi ubin di sepanjang dimensi tinggi. tile_sample_min_width (`int`, *opsional*) : Lebar minimum yang diperlukan agar sampel dipisahkan menjadi ubin di sepanjang dimensi lebar. tile_sample_stride_height (`int`, *opsional*) : Jumlah minimum tumpang tindih antara dua ubin vertikal yang berurutan. Hal ini untuk memastikan tidak ada artefak penataan ubin yang dihasilkan di sepanjang dimensi tinggi. tile_sample_stride_width (`int`, *opsional*) : Jarak (stride) antara dua tile horizontal yang berurutan. Hal ini untuk memastikan tidak ada artefak tiling yang dihasilkan di sepanjang dimensi lebar. Aktifkan decoding VAE bertile. Ketika opsi ini diaktifkan, VAE akan membagi tensor input menjadi beberapa tile untuk menghitung decoding dan encoding dalam beberapa langkah. Hal ini berguna untuk menghemat memori dalam jumlah besar dan memungkinkan pemrosesan gambar yang lebih besar. #### forward[[diffusers.AutoencoderKLCosmos.forward]] ```python forward(sample: torch.Tensor, sample_posterior: bool = False, return_dict: bool = True, generator: torch.Generator | None = None) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl_cosmos.py#L1074) **Parameter:** sample (`torch.Tensor`) : Sampel input. sample_posterior (`bool`, *opsional*, defaultnya `False`) : Apakah akan melakukan sampling dari posterior. return_dict (`bool`, *opsional*, defaultnya `True`) : Apakah akan mengembalikan `DecoderOutput` alih-alih tuple biasa. generator (`torch.Generator`, *opsional*) : Sebuah [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) untuk membuat sampling menjadi deterministik. **Mengembalikan:** `~models.vae.DecoderOutput` atau `tuple` Jika `return_dict` bernilai True, sebuah `~models.vae.DecoderOutput` akan dikembalikan, selain itu sebuah `tuple` biasa akan dikembalikan. ## AutoencoderKLOutput[[diffusers.models.modeling_outputs.AutoencoderKLOutput]] #### diffusers.models.modeling_outputs.AutoencoderKLOutput[[diffusers.models.modeling_outputs.AutoencoderKLOutput]] ```python diffusers.models.modeling_outputs.AutoencoderKLOutput(latent_dist: DiagonalGaussianDistribution) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/modeling_outputs.py#L7) **Parameter:** latent_dist (`DiagonalGaussianDistribution`) : Output hasil enkripsi dari `Encoder` yang direpresentasikan sebagai mean dan logvar dari `DiagonalGaussianDistribution`. `DiagonalGaussianDistribution` memungkinkan pengambilan sampel latents dari distribusi tersebut. Output dari metode encoding AutoencoderKL. ## DecoderOutput[[diffusers.models.autoencoders.vae.DecoderOutput]] #### diffusers.models.autoencoders.vae.DecoderOutput[[diffusers.models.autoencoders.vae.DecoderOutput]] ```python diffusers.models.autoencoders.vae.DecoderOutput(sample: Tensor, commit_loss: typing.Optional[torch.FloatTensor] = None) ``` [Sumber](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/vae.py#L49) **Parameter:** sample (`torch.Tensor` dengan bentuk `(batch_size, num_channels, height, width)`) : Sampel output yang didekodekan dari lapisan terakhir model. Output dari metode decoding.