Models - Multimodal
updated
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
Paper
• 2402.12226
• Published • 46
M2-CLIP: A Multimodal, Multi-task Adapting Framework for Video Action
Recognition
Paper
• 2401.11649
• Published • 3
Gen4Gen: Generative Data Pipeline for Generative Multi-Concept
Composition
Paper
• 2402.15504
• Published • 21
EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with
Audio2Video Diffusion Model under Weak Conditions
Paper
• 2402.17485
• Published • 193
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist
Autonomous Agents for Desktop and Web
Paper
• 2402.17553
• Published • 25
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction
Tuning
Paper
• 2309.02591
• Published • 16
ChatMusician: Understanding and Generating Music Intrinsically with LLM
Paper
• 2402.16153
• Published • 57
Valley: Video Assistant with Large Language model Enhanced abilitY
Paper
• 2306.07207
• Published • 3
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
Paper
• 2304.15010
• Published • 4
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Paper
• 2305.03726
• Published • 7
DeepSeek-VL: Towards Real-World Vision-Language Understanding
Paper
• 2403.05525
• Published • 49
Gemini 1.5: Unlocking multimodal understanding across millions of tokens
of context
Paper
• 2403.05530
• Published • 60
EVE: Efficient Vision-Language Pre-training with Masked Prediction and
Modality-Aware MoE
Paper
• 2308.11971
• Published • 2
Text Generation • Updated • 19.4k
• 391
google/siglip-so400m-patch14-384
Zero-Shot Image Classification • 0.9B • Updated • 756k
• 705
mistral-community/pixtral-12b-240910
Image-Text-to-Text • Updated • 21
• 380