We introduce Awesome Multimodal Modeling, a curated repository tracing the architectural evolution of multimodal intelligence—from foundational fusion to native omni-models.
🔹 Taxonomy & Evolution:
Traditional Multimodal Learning – Foundational work on representation, fusion, and alignment. Multimodal LLMs (MLLMs) – Architectures connecting vision encoders to LLMs for understanding. Unified Multimodal Models (UMMs) – Models unifying Understanding + Generation via Diffusion, Autoregressive, or Hybrid paradigms. Native Multimodal Models (NMMs) – Models trained from scratch on all modalities; contrasts early vs. late fusion under scaling laws. 💡 Key Distinction: UMMs unify tasks via generation heads; NMMs enforce interleaving through joint pre-training.
ScalingOpt is continuously evolving! We are steadily expanding the Community section with new content. For our Blog, we've launched by featuring work from Jianlin Su and are actively translating insightful posts from scientific communities into English to share on ScalingOpt (we'll keep curating excellent community blogs and providing English versions alongside the originals).
We operate under the Creative Commons Attribution-NonCommercial principle, sharing knowledge freely and openly. We welcome your ideas, suggestions, and feedback to help shape ScalingOpt's future.
If you find this initiative valuable, please consider following and starring the project to show your support. Thank you!