| ---
|
| license: cc-by-4.0
|
| tags:
|
| - pytorch
|
| - mocov3
|
| - classification
|
| ---
|
|
|
| # Mocov3 experiments: Classification
|
|
|
| ## Overview
|
|
|
| This repository is a compact, custom PyTorch implementation of **Mocov3** for **Classification**. The **huge** configuration is intended for code review, smoke tests, and small controlled experiments rather than as a production-ready pretrained release.
|
|
|
| The whole setup fits in a single module on purpose.
|
|
|
| ## Architecture
|
|
|
| | Item | Value |
|
| |---|---|
|
| | Architecture | Mocov3 |
|
| | Scale | huge |
|
| | Attention | sparse |
|
| | Fusion | concat mlp |
|
| | Activation | relu |
|
| | Normalization | rmsnorm |
|
|
|
| ## Repository status
|
|
|
| One Python module contains the model and a runnable example.
|
| Generated configuration lives in `config.json`; the recipe in `training_args.json`.
|
| The checkpoint is an initialization artifact and should not be read as a trained model.
|
| Scores are deliberately absent.
|
|
|
| ## Contact
|
|
|
| Questions or corrections: open a discussion in this repository.
|
|
|
| ## Limitations
|
|
|
| The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
|
|
|
| ## Roadmap
|
|
|
| Next steps, in no particular order: a cleaner entry point, input-shape docs, and a small evaluation script if time permits.
|
|
|
| ## Default experiment recipe
|
|
|
| Defaults: **adafactor** optimizer, **exponential** schedule. Treat these as starting points; a real run should sweep learning rate and batch size before drawing conclusions.
|
|
|
| ## Files
|
|
|
| | File | Purpose |
|
| |---|---|
|
| | `train.py` | primary artifact |
|
| | `README.md` | this documentation |
|
| | `config.json` | architecture configuration |
|
| | `training_args.json` | default experiment settings |
|
| | `model.safetensors` | initialization checkpoint |
|
|
|
| ## License
|
|
|
| Distributed under the **cc-by-4.0** license. External datasets keep their own terms — check them separately.
|
|
|
