|
Download README.md from FlorindoDev/OutfitTransformer-weights: direct link, hf CLI and curl.
- Browser
- Download file 30.9 kB
-
https://huggingface.co/FlorindoDev/OutfitTransformer-weights/resolve/main/README.md
- Command line
-
hf download hf://FlorindoDev/OutfitTransformer-weights/README.md
-
curl -L -o README.md https://huggingface.co/FlorindoDev/OutfitTransformer-weights/resolve/main/README.md
30.9 kB
| language: | |
| - en | |
| - it | |
| tags: | |
| - pytorch | |
| - outfit-transformer | |
| - fashion | |
| - compatibility-prediction | |
| - complementary-item-retrieval | |
| - embeddings | |
| > [!NOTE] | |
| > This README is available in English below and in [Italian / italiano](#versione-italiana). | |
| > La [versione italiana](#versione-italiana) si trova nella seconda parte del documento. | |
| > [!IMPORTANT] | |
| > These weights were trained and used with my **OutfitTransformer** model, available in the [FlorindoDev/OutfitTransformer](https://github.com/FlorindoDev/OutfitTransformer) repository. To use them, refer to that project's code, architecture, and instructions. | |
| # OutfitTransformer: Weights and precomputed embeddings | |
| This repository contains checkpoints trained with my implementation of | |
| OutfitTransformer and **precomputed embeddings** for the project's training | |
| and evaluation workflows. | |
| The source code, model configuration, and usage instructions are available on | |
| [GitHub](https://github.com/FlorindoDev/OutfitTransformer). | |
| It includes the **cp_precomputed-flashionclip** (Compatibility Prediction) and | |
| **cir_precomputed-flashionclip** (Complementary Item Retrieval) runs, both on the Polyvore | |
| `nondisjoint` subset, together with train, validation, and test embeddings. | |
| ## Contents and organization | |
| Artifacts are organized as follows. Each run contains `best.pt`, `config.json`, | |
| individual epoch checkpoints in `epochs/`, and plots in `plots/`. | |
| ```text | |
| . | |
| ├── README.md | |
| ├── .gitattributes | |
| ├── checkpoints/ | |
| │ └── nondisjoint/ | |
| │ ├── cp_precomputed-flashionclip/ | |
| │ └── cir_precomputed-flashionclip/ | |
| └── precomputed_embeddings/ | |
| └── nondisjoint/ | |
| └── fashionCLIP/ | |
| ├── train/ # manifest.json + shard-*.pt | |
| ├── validation/ # manifest.json + shard-*.pt | |
| └── test/ # manifest.json + shard-*.pt | |
| ``` | |
| ## Parameters and hyperparameters of the published runs | |
| The following values come from the saved configurations for | |
| [cp_precomputed-flashionclip](checkpoints/nondisjoint/cp_precomputed-flashionclip/config.json) and | |
| [cir_precomputed-flashionclip](checkpoints/nondisjoint/cir_precomputed-flashionclip/config.json), compared | |
| with `run_config` in their respective `best.pt` files and latest available | |
| checkpoints. Descriptions follow the project documentation; the values shown | |
| are those of the published experiments. `null` indicates an unset value; | |
| **not recorded** indicates a field absent from the saved configuration. | |
| ### Checkpoints and recorded progress | |
| | Parameter | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Run folder | `checkpoints/nondisjoint/cp_precomputed-flashionclip/` | `checkpoints/nondisjoint/cir_precomputed-flashionclip/` | | |
| | Checkpoint format version (`checkpoint_schema_version`) | `2` | `1` | | |
| | Configured maximum epochs (`training.epochs`) | `200` | `200` | | |
| | Available epoch checkpoints | `81`, epochs `1–81` | `42`, epochs `1–42` | | |
| | Latest available checkpoint | `epochs/cp_epoch_081.pt` | `epochs/cir_epoch_042.pt` | | |
| | Epoch of `best.pt` | `73` | `17` | | |
| | Selection metric (`training.best_metric`) | `val_auc`, maximize | `val_fitb_accuracy`, maximize | | |
| | Metric value in the best checkpoint | AUC `0.95082516` | FITB accuracy `0.6854` (`68.54%`) | | |
| | Training examples in the best epoch | `106612` | `53248` | | |
| | Validation examples in the best epoch | `10000` | `5000` | | |
| The 200 epochs and the total number of optimizer steps listed below describe | |
| the configured plan. Available checkpoints document training through epochs | |
| 81 and 42 respectively; they do not establish that all 200 epochs were completed. | |
| ### Dataset and features | |
| The values in this table are shared by both runs. | |
| | Parameter | Recorded value | | |
| | --- | --- | | |
| | Dataset (`dataset.name`) | `polyvore` | | |
| | Dataset identifier (`dataset.id`) | `mvasil/polyvore-outfits` | | |
| | Subset (`dataset.subset`) | `nondisjoint` | | |
| | Dataset directory during training (`dataset.dataset_root`) | `datasets/polyvore-outfits` | | |
| | Custom dataset cache (`dataset.cache_dir`) | `null` | | |
| | Embedding directory during training (`dataset.embedding_root`) | `precomputed_embeddings/patrickjohncyh-fashion-clip` | | |
| | Feature mode (`dataset.feature_mode`, `features.mode`) | `precomputed` | | |
| | Use of precomputed embeddings (`features.precomputed`) | `true` | | |
| | Encoder in the configuration (`features.encoder`) | `from_embedding_manifest` | | |
| | Encoder identified in the manifests | `patrickjohncyh/fashion-clip` | | |
| | Trainable features (`features.trainable`) | `false`: the encoders are not trained in these runs | | |
| The recorded embedding path is the one used during training. In this repository, | |
| the caches are stored directly under `precomputed_embeddings/nondisjoint/fashionCLIP/<split>/`. | |
| For training, arrange the downloaded splits as `<embedding-root>/nondisjoint/<split>/` | |
| and set `--embedding-root` to that root directory. | |
| ### Architecture | |
| For CP, Transformer keys are under `model`; for CIR, they are under | |
| `model.transformer`. The total input dimension is derived by concatenating | |
| the two modalities. | |
| | Parameter | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Dimension per modality (`modality_embedding_dim`) | `512` | `512` | | |
| | Total input / Transformer dimension | `1024 = 512 + 512` | `1024 = 512 + 512` | | |
| | Number of layers (`layers`) | `6` | `6` | | |
| | Attention heads (`attention_heads`) | `16` | `16` | | |
| | Feed-forward dimension (`feedforward_dim`) | `2024` | `2024` | | |
| | Activation (`activation`) | `mish` | `mish` | | |
| | Dropout (`dropout`) | `0.3` | `0.3` | | |
| | Pre-norm (`norm_first`) | `true` | `true` | | |
| | LayerNorm epsilon (`layer_norm_epsilon`) | `1e-5` | `1e-5` | | |
| | L2 normalization epsilon (`normalization_epsilon`) | `1e-12` | `1e-12` | | |
| | Embedding initialization standard deviation (`embedding_initialization_std`) | `0.02` | `0.02` | | |
| | Maximum number of items (`max_items`) | `16` | `16` | | |
| | Additional final LayerNorm, verified in the weights | Present: `cp.encoder.norm.*` | Absent: no `cir.encoder.norm.*` keys | | |
| | Retrieval embedding dimension (`model.cir.embedding_dim`) | Not applicable | `128` | | |
| | L2 normalization of final retrieval embeddings (`model.cir.normalize_embeddings`) | Not applicable | `false` | | |
| | Category embedding (`model.use_category_embedding`) | Not applicable | `false` | | |
| ### Training and loss function | |
| | Parameter | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Microbatch (`training.batch_size` / `training.batch_size_per_process`) | `512` outfits | `64` outfits per process | | |
| | Gradient accumulation (`training.gradient_accumulation_steps`) | `4` microbatches | `4` microbatches | | |
| | Effective batch size (`training.effective_batch_size` / `training.effective_batch_size_global`) | `2048` | `256` global | | |
| | Effective batch size per process (`training.effective_batch_size_per_process`) | Not recorded separately | `256` | | |
| | Optimizer (`training.optimizer`) | `AdamW` | `AdamW` | | |
| | Configured / maximum learning rate (`training.learning_rate`) | `2e-5` | `2e-5` | | |
| | Weight decay (`training.weight_decay`) | `0.01` | `0.01` | | |
| | Gradient clipping, maximum norm (`training.max_grad_norm`) | `1.0` | `1.0` | | |
| | Seed (`training.seed`) | `42` | `42` | | |
| | Loss (`training.loss`) | `FocalLoss` | `InBatchTripletMarginLoss` | | |
| | Loss reduction (`training.focal_reduction` / `training.loss_reduction`, `model.cir.loss_reduction`) | `mean` | `mean` | | |
| | Focal alpha (`training.focal_alpha`) | `0.5` | Not applicable | | |
| | Focal gamma (`training.focal_gamma`) | `2.0` | Not applicable | | |
| | Triplet margin (`training.triplet_margin`, `model.cir.triplet_margin`) | Not applicable | `2.0` | | |
| | Negative sampling (`training.negative_sampling`) | Not applicable | `in_batch` | | |
| | Target sampling (`training.target_sampling`) | Not applicable | `random_item_per_outfit_access` | | |
| | Early stopping patience (`training.early_stopping_patience`) | `null`: disabled | `null`: disabled | | |
| | Minimum improvement (`training.early_stopping_min_delta`) | `0.0` | `0.0` | | |
| | Resume weights (`training.resume_weights`) | `null` | `null` | | |
| | Initialization from CP (`training.pretrained_cp_weights`) | Not applicable | `best.pt` | | |
| | Optimizer steps per epoch (`training.optimizer_steps_per_epoch`) | `53` | `208` | | |
| | Total planned optimizer steps (`training.total_optimizer_steps`) | `10600` | `41600` | | |
| For CIR, the target is an item chosen at random on every access to an outfit. | |
| As described in the training README, the loss selects the hardest negative | |
| among the positives from the other rows in the same microbatch: accumulating | |
| up to 256 examples does not expand the negative pool of the 64-example microbatch. | |
| CIR initialization records only the name `best.pt` of the source CP checkpoint, | |
| without its full path or hash. This alone does not uniquely identify that file | |
| as the `cp_precomputed-flashionclip/best.pt` published here. | |
| ### Scheduler | |
| All fields in this table are saved under `training.scheduler` and have the same | |
| values in both runs. | |
| | Parameter | Value | | |
| | --- | --- | | |
| | Scheduler (`name`) | `OneCycleLR` | | |
| | Update frequency (`step`) | `optimizer_step`: after each optimizer update | | |
| | Maximum learning rate (`max_lr`) | `2e-5` | | |
| | Initial fraction of the cycle (`pct_start`) | `0.3` | | |
| | Annealing strategy (`anneal_strategy`) | `cos` | | |
| | Initial LR factor (`div_factor`) | `25.0` | | |
| | Minimum LR factor (`final_div_factor`) | `10000.0` | | |
| ### Runtime | |
| | Parameter | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Requested device (`runtime.requested_device`) | `auto` | `auto` | | |
| | Resolved device (`runtime.resolved_device`) | `cuda` | `cuda` | | |
| | DataLoader workers (`runtime.num_workers`) | `0` | `0` | | |
| | Pinned memory (`runtime.pin_memory`) | `false` | `false` | | |
| | Logging frequency (`runtime.log_every`) | Every `10` microbatches | Every `10` microbatches | | |
| | Distributed training (`runtime.ddp`) | Not recorded | `false` | | |
| | Number of processes (`runtime.world_size`) | Not recorded | `1` | | |
| | Distributed backend (`runtime.distributed_backend`) | Not recorded | `null` | | |
| | Mixed precision (`runtime.mixed_precision`) | Not recorded | `false` | | |
| The tables cover all fields present in the two `config.json` files. Implicit | |
| settings that were not serialized, such as AdamW betas and epsilon, and details | |
| such as the GPU model and software versions are not documented by the inspected | |
| artifacts. | |
| ## Checkpoints | |
| Weights are saved in **PyTorch `.pt` format**. In the project's current format, | |
| each checkpoint contains `model_state_dict`, `run_config`, the epoch number, | |
| metrics, training history, and best-model selection information. `config.json` | |
| stores a readable version of the experiment configuration. | |
| `best.pt` is the checkpoint selected during training according to the configured | |
| metric. Files in `epochs/`, when available, allow other epochs from the run to | |
| be used. Current checkpoints do not include optimizer or scheduler state: | |
| resuming in this project initializes a new run from the saved weights. | |
| To load a checkpoint, use the loaders and commands from the source repository | |
| with the configuration and architecture version used during training. It is | |
| also useful to record the Git commit of the code used for each experiment. | |
| The CP checkpoint published here contains `cp.encoder.norm.*`, which is absent | |
| from the current CP architecture. Handling these checkpoints for CIR | |
| initialization is described in the | |
| [CIR README](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CIR/README.md). | |
| Uploading to the Hub preserves the project's format; it does not automatically | |
| add support for `transformers.AutoModel.from_pretrained()`. | |
| ## Precomputed embeddings | |
| Embeddings are numerical representations of items, generated using the | |
| [project's precomputation procedure](https://github.com/FlorindoDev/OutfitTransformer/blob/main/scripts/README.md). | |
| They can be reused in workflows that support precomputed features, avoiding | |
| the need to run the encoders again for each item. | |
| Each cache includes **`manifest.json` and all `shard-*.pt` files listed in the | |
| manifest**, kept in the same folder. Shards contain `schema_version`, the | |
| `item_ids` identifiers, and the `embeddings` tensor; the manifest describes | |
| the encoder, dataset, subset, split, dimensions, precision, fingerprint, and shards. | |
| In the current procedure, each embedding concatenates the image and text | |
| representations, each normalized separately with L2 normalization. With | |
| FashionCLIP and the default configuration, each vector has 1024 components: | |
| 512 visual and 512 textual. The actual cache values are documented in its manifest. | |
| ### Parameters of the published caches | |
| | Parameter | Value in the manifests | | |
| | --- | --- | | |
| | Format version (`schema_version`) | `2` | | |
| | Dataset / subset | `mvasil/polyvore-outfits` / `nondisjoint` | | |
| | Encoder model (`encoder.model_name`) | `patrickjohncyh/fashion-clip` | | |
| | Visual encoder (`encoder.visual_encoder`) | `FashionCLIPVisualEncoder` | | |
| | Text encoder (`encoder.text_encoder`) | `FashionCLIPTextEncoder` | | |
| | Dimension per modality (`encoder.modality_dim`) | `512` | | |
| | Final dimension (`embedding_dim`) | `1024` | | |
| | Aggregation (`encoder.aggregation`) | `concat_visual_then_text` | | |
| | Normalization (`encoder.normalization`) | `l2_per_modality` | | |
| | Saved precision (`dtype`) | `float32` | | |
| | Item count limit (`limit`) | `null` | | |
| | Visual / text encoder commit (`encoder.visual_commit`, `encoder.text_commit`) | `null` / `null` | | |
| | Split | Items (`count`) | Shards | Manifest | | |
| | --- | --- | --- | --- | | |
| | Train | `204679` | `21` | [train/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/train/manifest.json) | | |
| | Validation | `25132` | `3` | [validation/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/validation/manifest.json) | | |
| | Test | `47854` | `5` | [test/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/test/manifest.json) | | |
| Each shard contains 10000 items, except the last shard in each split: 4679 for | |
| train, 5132 for validation, and 7854 for test. These counts refer to items with | |
| embeddings, not outfits or training examples. | |
| Train and validation share the `model_fingerprint` | |
| `dfcff17f9fb9335b22792d610cb7e4ba8cd73afa2954b725aa5e2d993bc10c27`. | |
| The test cache instead records | |
| `b00590d05bc4af1a3a5316ecb65dabea7315fa93adf0dcbdbec0b8399ec1e788`. | |
| The encoder name and dimensions match, but the fingerprints differ: these | |
| metadata do not establish that the precomputation configuration is identical | |
| between test and the other splits. | |
| Use embeddings consistent with the checkpoint and the required dataset, | |
| subset, and split. Training workflows also verify that encoder fingerprints | |
| match between train and validation caches. Caches contain identifiers and | |
| vectors; the data needed to construct outfits, labels, and queries are still | |
| those required by the project. | |
| ## References | |
| - [OutfitTransformer source code](https://github.com/FlorindoDev/OutfitTransformer) | |
| - [CP training](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CP/README.md) | |
| - [CIR training](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CIR/README.md) | |
| - [Embedding precomputation](https://github.com/FlorindoDev/OutfitTransformer/blob/main/scripts/README.md) | |
| --- | |
| <a id="versione-italiana"></a> | |
| > [!IMPORTANT] | |
| > Questi sono pesi allenati e utilizzati con il mio modello **OutfitTransformer**, disponibile nella repository [FlorindoDev/OutfitTransformer](https://github.com/FlorindoDev/OutfitTransformer). Per utilizzarli, fai riferimento al codice, all'architettura e alle istruzioni di quel progetto. | |
| # OutfitTransformer: Pesi ed embedding precomputati | |
| Questa repository raccoglie i checkpoint allenati con la mia implementazione di | |
| OutfitTransformer e gli **embedding precomputati** utilizzabili nei flussi di | |
| training e valutazione del progetto. | |
| Il codice sorgente, la configurazione del modello e le istruzioni di utilizzo sono | |
| disponibili su [GitHub](https://github.com/FlorindoDev/OutfitTransformer). | |
| Sono inclusi i run **cp_precomputed-flashionclip** (Compatibility Prediction) e | |
| **cir_precomputed-flashionclip** (Complementary Item Retrieval), entrambi sul subset | |
| `nondisjoint` di Polyvore, insieme agli embedding di train, validation e test. | |
| ## Contenuto e organizzazione | |
| Gli artefatti sono organizzati come segue. Ogni run contiene `best.pt`, | |
| `config.json`, i checkpoint delle singole epoche in `epochs/` e i grafici in | |
| `plots/`. | |
| ```text | |
| . | |
| ├── README.md | |
| ├── .gitattributes | |
| ├── checkpoints/ | |
| │ └── nondisjoint/ | |
| │ ├── cp_precomputed-flashionclip/ | |
| │ └── cir_precomputed-flashionclip/ | |
| └── precomputed_embeddings/ | |
| └── nondisjoint/ | |
| └── fashionCLIP/ | |
| ├── train/ # manifest.json + shard-*.pt | |
| ├── validation/ # manifest.json + shard-*.pt | |
| └── test/ # manifest.json + shard-*.pt | |
| ``` | |
| ## Parametri e iperparametri dei run pubblicati | |
| I valori seguenti provengono dalle configurazioni salvate nei run | |
| [cp_precomputed-flashionclip](checkpoints/nondisjoint/cp_precomputed-flashionclip/config.json) e | |
| [cir_precomputed-flashionclip](checkpoints/nondisjoint/cir_precomputed-flashionclip/config.json), confrontate | |
| con `run_config` nei rispettivi `best.pt` e nell'ultimo checkpoint disponibile. | |
| Le descrizioni seguono la documentazione del progetto; i valori riportati sono | |
| quelli degli esperimenti pubblicati. `null` indica un valore non impostato; | |
| **non registrato** indica un campo assente dalla configurazione salvata. | |
| ### Checkpoint e avanzamento registrato | |
| | Parametro | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Cartella del run | `checkpoints/nondisjoint/cp_precomputed-flashionclip/` | `checkpoints/nondisjoint/cir_precomputed-flashionclip/` | | |
| | Versione del formato checkpoint (`checkpoint_schema_version`) | `2` | `1` | | |
| | Epoche massime configurate (`training.epochs`) | `200` | `200` | | |
| | Checkpoint di epoca disponibili | `81`, epoche `1–81` | `42`, epoche `1–42` | | |
| | Ultimo checkpoint disponibile | `epochs/cp_epoch_081.pt` | `epochs/cir_epoch_042.pt` | | |
| | Epoca di `best.pt` | `73` | `17` | | |
| | Metrica di selezione (`training.best_metric`) | `val_auc`, da massimizzare | `val_fitb_accuracy`, da massimizzare | | |
| | Valore della metrica nel best | AUC `0.95082516` | FITB accuracy `0.6854` (`68.54%`) | | |
| | Esempi train nell'epoca del best | `106612` | `53248` | | |
| | Esempi validation nell'epoca del best | `10000` | `5000` | | |
| Le 200 epoche e il numero totale di optimizer step riportato sotto descrivono il | |
| piano configurato. I checkpoint disponibili documentano il training fino alle | |
| epoche 81 e 42 rispettivamente; non attestano il completamento delle 200 epoche. | |
| ### Dataset e feature | |
| I valori di questa tabella sono comuni ai due run. | |
| | Parametro | Valore registrato | | |
| | --- | --- | | |
| | Dataset (`dataset.name`) | `polyvore` | | |
| | Identificativo dataset (`dataset.id`) | `mvasil/polyvore-outfits` | | |
| | Subset (`dataset.subset`) | `nondisjoint` | | |
| | Directory dataset nel training (`dataset.dataset_root`) | `datasets/polyvore-outfits` | | |
| | Cache dataset personalizzata (`dataset.cache_dir`) | `null` | | |
| | Directory embedding nel training (`dataset.embedding_root`) | `precomputed_embeddings/patrickjohncyh-fashion-clip` | | |
| | Modalità feature (`dataset.feature_mode`, `features.mode`) | `precomputed` | | |
| | Uso di embedding precomputati (`features.precomputed`) | `true` | | |
| | Encoder nella configurazione (`features.encoder`) | `from_embedding_manifest` | | |
| | Encoder identificato nei manifest | `patrickjohncyh/fashion-clip` | | |
| | Feature allenabili (`features.trainable`) | `false`: gli encoder non vengono allenati in questi run | | |
| Il percorso degli embedding registrato è quello usato durante il training. | |
| In questa repository le cache si trovano direttamente in | |
| `precomputed_embeddings/nondisjoint/fashionCLIP/<split>/`. Per il training, disponi gli split scaricati come | |
| `<embedding-root>/nondisjoint/<split>/` e imposta `--embedding-root` su quella directory radice. | |
| ### Architettura | |
| Per CP le chiavi del Transformer sono in `model`; per CIR sono in | |
| `model.transformer`. La dimensione totale di input è derivata dalla | |
| concatenazione delle due modalità. | |
| | Parametro | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Dimensione per modalità (`modality_embedding_dim`) | `512` | `512` | | |
| | Dimensione totale di input / Transformer | `1024 = 512 + 512` | `1024 = 512 + 512` | | |
| | Numero di layer (`layers`) | `6` | `6` | | |
| | Teste di attenzione (`attention_heads`) | `16` | `16` | | |
| | Dimensione feed-forward (`feedforward_dim`) | `2024` | `2024` | | |
| | Attivazione (`activation`) | `mish` | `mish` | | |
| | Dropout (`dropout`) | `0.3` | `0.3` | | |
| | Pre-norm (`norm_first`) | `true` | `true` | | |
| | Epsilon LayerNorm (`layer_norm_epsilon`) | `1e-5` | `1e-5` | | |
| | Epsilon normalizzazione L2 (`normalization_epsilon`) | `1e-12` | `1e-12` | | |
| | Deviazione standard inizializzazione embedding (`embedding_initialization_std`) | `0.02` | `0.02` | | |
| | Numero massimo di item (`max_items`) | `16` | `16` | | |
| | LayerNorm finale aggiuntiva, verificata nei pesi | Presente: `cp.encoder.norm.*` | Assente: nessuna chiave `cir.encoder.norm.*` | | |
| | Dimensione embedding di retrieval (`model.cir.embedding_dim`) | Non applicabile | `128` | | |
| | Normalizzazione L2 degli embedding finali di retrieval (`model.cir.normalize_embeddings`) | Non applicabile | `false` | | |
| | Embedding di categoria (`model.use_category_embedding`) | Non applicabile | `false` | | |
| ### Training e funzione di loss | |
| | Parametro | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Microbatch (`training.batch_size` / `training.batch_size_per_process`) | `512` outfit | `64` outfit per processo | | |
| | Accumulo gradienti (`training.gradient_accumulation_steps`) | `4` microbatch | `4` microbatch | | |
| | Batch effettivo (`training.effective_batch_size` / `training.effective_batch_size_global`) | `2048` | `256` globale | | |
| | Batch effettivo per processo (`training.effective_batch_size_per_process`) | Non registrato separatamente | `256` | | |
| | Ottimizzatore (`training.optimizer`) | `AdamW` | `AdamW` | | |
| | Learning rate configurato / massimo (`training.learning_rate`) | `2e-5` | `2e-5` | | |
| | Weight decay (`training.weight_decay`) | `0.01` | `0.01` | | |
| | Gradient clipping, norma massima (`training.max_grad_norm`) | `1.0` | `1.0` | | |
| | Seed (`training.seed`) | `42` | `42` | | |
| | Loss (`training.loss`) | `FocalLoss` | `InBatchTripletMarginLoss` | | |
| | Riduzione della loss (`training.focal_reduction` / `training.loss_reduction`, `model.cir.loss_reduction`) | `mean` | `mean` | | |
| | Focal alpha (`training.focal_alpha`) | `0.5` | Non applicabile | | |
| | Focal gamma (`training.focal_gamma`) | `2.0` | Non applicabile | | |
| | Margine triplet (`training.triplet_margin`, `model.cir.triplet_margin`) | Non applicabile | `2.0` | | |
| | Campionamento negativi (`training.negative_sampling`) | Non applicabile | `in_batch` | | |
| | Campionamento target (`training.target_sampling`) | Non applicabile | `random_item_per_outfit_access` | | |
| | Early stopping, patience (`training.early_stopping_patience`) | `null`: disabilitato | `null`: disabilitato | | |
| | Miglioramento minimo (`training.early_stopping_min_delta`) | `0.0` | `0.0` | | |
| | Pesi per resume (`training.resume_weights`) | `null` | `null` | | |
| | Inizializzazione da CP (`training.pretrained_cp_weights`) | Non applicabile | `best.pt` | | |
| | Optimizer step per epoca (`training.optimizer_steps_per_epoch`) | `53` | `208` | | |
| | Optimizer step totali pianificati (`training.total_optimizer_steps`) | `10600` | `41600` | | |
| Per CIR il target è un item scelto casualmente a ogni accesso all'outfit. Come | |
| descritto nel README del training, la loss seleziona il negativo più difficile | |
| tra i positivi delle altre righe dello stesso microbatch: l'accumulo fino a 256 | |
| esempi non amplia il gruppo di negativi del microbatch da 64. | |
| L'inizializzazione CIR registra soltanto il nome `best.pt` del checkpoint CP | |
| sorgente, senza percorso completo o hash. Questo dato da solo non identifica | |
| univocamente il file come il `cp_precomputed-flashionclip/best.pt` qui pubblicato. | |
| ### Scheduler | |
| Tutti i campi di questa tabella sono salvati in `training.scheduler` e hanno lo | |
| stesso valore nei due run. | |
| | Parametro | Valore | | |
| | --- | --- | | |
| | Scheduler (`name`) | `OneCycleLR` | | |
| | Frequenza di aggiornamento (`step`) | `optimizer_step`: dopo ogni aggiornamento dell'ottimizzatore | | |
| | Learning rate massimo (`max_lr`) | `2e-5` | | |
| | Frazione iniziale del ciclo (`pct_start`) | `0.3` | | |
| | Strategia di annealing (`anneal_strategy`) | `cos` | | |
| | Fattore LR iniziale (`div_factor`) | `25.0` | | |
| | Fattore LR minimo (`final_div_factor`) | `10000.0` | | |
| ### Runtime | |
| | Parametro | cp_precomputed-flashionclip | cir_precomputed-flashionclip | | |
| | --- | --- | --- | | |
| | Dispositivo richiesto (`runtime.requested_device`) | `auto` | `auto` | | |
| | Dispositivo risolto (`runtime.resolved_device`) | `cuda` | `cuda` | | |
| | Worker del DataLoader (`runtime.num_workers`) | `0` | `0` | | |
| | Pinned memory (`runtime.pin_memory`) | `false` | `false` | | |
| | Frequenza log (`runtime.log_every`) | Ogni `10` microbatch | Ogni `10` microbatch | | |
| | Training distribuito (`runtime.ddp`) | Non registrato | `false` | | |
| | Numero processi (`runtime.world_size`) | Non registrato | `1` | | |
| | Backend distribuito (`runtime.distributed_backend`) | Non registrato | `null` | | |
| | Mixed precision (`runtime.mixed_precision`) | Non registrata | `false` | | |
| Le tabelle coprono tutti i campi presenti nei due `config.json`. Impostazioni | |
| implicite non serializzate, come i beta e l'epsilon di AdamW, e dettagli quali | |
| modello di GPU e versioni software non sono documentati dagli artefatti letti. | |
| ## Checkpoint | |
| I pesi sono salvati in formato **PyTorch `.pt`**. Nel formato attuale del progetto, | |
| ogni checkpoint contiene `model_state_dict`, `run_config`, il numero dell'epoca, | |
| le metriche, la cronologia del training e le informazioni sulla selezione del | |
| modello migliore. `config.json` conserva una versione leggibile della | |
| configurazione dell'esperimento. | |
| `best.pt` è il checkpoint selezionato dal training secondo la metrica configurata. | |
| Gli eventuali file in `epochs/` permettono di utilizzare altre epoche del run. | |
| I checkpoint attuali non includono lo stato di optimizer e scheduler: il resume | |
| del progetto inizializza un nuovo run a partire dai pesi salvati. | |
| Per caricare un checkpoint, usa i loader e i comandi della repository sorgente, | |
| con la configurazione e la versione dell'architettura usate durante il training. | |
| Per ciascun esperimento è utile riportare anche il commit Git del codice usato. | |
| Il checkpoint CP qui pubblicato contiene `cp.encoder.norm.*`, assente | |
| nell'architettura CP attuale. La gestione di questi checkpoint per | |
| l'inizializzazione di CIR è descritta nel | |
| [README CIR](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CIR/README.md). | |
| Il caricamento sul Hub conserva il formato del progetto; non aggiunge | |
| automaticamente il supporto a `transformers.AutoModel.from_pretrained()`. | |
| ## Embedding precomputati | |
| Gli embedding sono rappresentazioni numeriche degli articoli, generate con la | |
| [procedura di precomputazione del progetto](https://github.com/FlorindoDev/OutfitTransformer/blob/main/scripts/README.md). | |
| Possono essere riutilizzati nei flussi che supportano feature precomputate, | |
| evitando di eseguire nuovamente gli encoder per ogni articolo. | |
| Ogni cache comprende **`manifest.json` e tutti gli `shard-*.pt` elencati nel | |
| manifest**, mantenuti nella stessa cartella. Gli shard contengono `schema_version`, | |
| gli identificativi `item_ids` e il tensore `embeddings`; il manifest descrive | |
| encoder, dataset, subset, split, dimensioni, precisione, fingerprint e shard. | |
| Nella procedura attuale, ogni embedding concatena le rappresentazioni di immagine | |
| e testo, normalizzate L2 separatamente. Con FashionCLIP e la configurazione | |
| predefinita, ogni vettore ha 1024 componenti: 512 visuali e 512 testuali. | |
| I valori effettivi della cache sono documentati nel relativo manifest. | |
| ### Parametri delle cache pubblicate | |
| | Parametro | Valore nei manifest | | |
| | --- | --- | | |
| | Versione del formato (`schema_version`) | `2` | | |
| | Dataset / subset | `mvasil/polyvore-outfits` / `nondisjoint` | | |
| | Modello encoder (`encoder.model_name`) | `patrickjohncyh/fashion-clip` | | |
| | Encoder visuale (`encoder.visual_encoder`) | `FashionCLIPVisualEncoder` | | |
| | Encoder testuale (`encoder.text_encoder`) | `FashionCLIPTextEncoder` | | |
| | Dimensione per modalità (`encoder.modality_dim`) | `512` | | |
| | Dimensione finale (`embedding_dim`) | `1024` | | |
| | Aggregazione (`encoder.aggregation`) | `concat_visual_then_text` | | |
| | Normalizzazione (`encoder.normalization`) | `l2_per_modality` | | |
| | Precisione salvata (`dtype`) | `float32` | | |
| | Limite al numero di articoli (`limit`) | `null` | | |
| | Commit encoder visuale / testuale (`encoder.visual_commit`, `encoder.text_commit`) | `null` / `null` | | |
| | Split | Articoli (`count`) | Shard | Manifest | | |
| | --- | --- | --- | --- | | |
| | Train | `204679` | `21` | [train/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/train/manifest.json) | | |
| | Validation | `25132` | `3` | [validation/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/validation/manifest.json) | | |
| | Test | `47854` | `5` | [test/manifest.json](precomputed_embeddings/nondisjoint/fashionCLIP/test/manifest.json) | | |
| Gli shard contengono 10000 articoli ciascuno, tranne gli ultimi: 4679 per train, | |
| 5132 per validation e 7854 per test. I conteggi indicano articoli con embedding, | |
| non outfit o esempi di training. | |
| Train e validation condividono il `model_fingerprint` | |
| `dfcff17f9fb9335b22792d610cb7e4ba8cd73afa2954b725aa5e2d993bc10c27`. | |
| La cache test registra invece | |
| `b00590d05bc4af1a3a5316ecb65dabea7315fa93adf0dcbdbec0b8399ec1e788`. | |
| Nome encoder e dimensioni coincidono, ma i fingerprint differiscono: questi | |
| metadati non attestano l'identità della configurazione di precomputazione tra | |
| test e gli altri split. | |
| Usa embedding coerenti con il checkpoint e con il dataset, il subset e lo split | |
| richiesti. I flussi di training verificano anche la corrispondenza del fingerprint | |
| degli encoder tra le cache di train e validation. Le cache contengono gli | |
| identificativi e i vettori; i dati necessari a costruire outfit, etichette e query | |
| restano quelli richiesti dal progetto. | |
| ## Riferimenti | |
| - [Codice sorgente OutfitTransformer](https://github.com/FlorindoDev/OutfitTransformer) | |
| - [Training CP](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CP/README.md) | |
| - [Training CIR](https://github.com/FlorindoDev/OutfitTransformer/blob/main/training/CIR/README.md) | |
| - [Precomputazione degli embedding](https://github.com/FlorindoDev/OutfitTransformer/blob/main/scripts/README.md) | |