Jiaqi-hkust (Jiaqi Tang)

reacted to their post with 🧠 about 10 hours ago

Post

84

We reshaped how VLMs connect vision to language!

One simple architectural change → dramatically better visual information injection into the LLM.

Accepted at NeurIPS. Check it out 👇

📄 https://arxiv.org/abs/2611.06844
💻 https://github.com/YingyingYan1/DGP

#NeurIPS #VLM #MLLM #ViT

reacted to their post with ❤️➕😎👀🔥 3 days ago

Post

84

We reshaped how VLMs connect vision to language!

One simple architectural change → dramatically better visual information injection into the LLM.

Accepted at NeurIPS. Check it out 👇

📄 https://arxiv.org/abs/2611.06844
💻 https://github.com/YingyingYan1/DGP

#NeurIPS #VLM #MLLM #ViT

posted an update 3 days ago

Post

84

We reshaped how VLMs connect vision to language!

One simple architectural change → dramatically better visual information injection into the LLM.

Accepted at NeurIPS. Check it out 👇

📄 https://arxiv.org/abs/2611.06844
💻 https://github.com/YingyingYan1/DGP

#NeurIPS #VLM #MLLM #ViT

reacted to their post with ❤️👀🔥 about 1 month ago

Post

3,202

We open-sourced Game-Creation-Agents — an agentic system that turns a one-line game idea into a playable browser game. 🎮

Why it matters:
- It lowers the barrier: you don’t need a full studio or deep engine skills to start.
- It compresses prototyping from weeks to hours.
- It lets creators test more ideas, faster, and focus on what’s actually fun.
- It brings game creation closer to a conversation — and makes it accessible to more people.

This isn’t just about generating code. It’s about giving more people the power to bring their ideas to life.

Try it:
https://github.com/Tencent/Game-Creation-Agents

#AI #GameDev #OpenSource

posted an update about 1 month ago

Post

3,202

We open-sourced Game-Creation-Agents — an agentic system that turns a one-line game idea into a playable browser game. 🎮

Why it matters:
- It lowers the barrier: you don’t need a full studio or deep engine skills to start.
- It compresses prototyping from weeks to hours.
- It lets creators test more ideas, faster, and focus on what’s actually fun.
- It brings game creation closer to a conversation — and makes it accessible to more people.

This isn’t just about generating code. It’s about giving more people the power to bring their ideas to life.

Try it:
https://github.com/Tencent/Game-Creation-Agents

#AI #GameDev #OpenSource

reacted to their post with 🤗➕👀🚀🔥 3 months ago

Post

2,429

🧠 Remember-R1: Our fix for MLLMs forgetting the image during long reasoning

We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the image—and start hallucinating based on their own text.

So we built Remember‑R1, a simple RL framework that directly supervises visual attention on the original reasoning trajectory—no inference overhead, no proxy tasks.

We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.

Results across 7 benchmarks and 2 model sizes: better reasoning, and—more importantly—visual attention decays much more slowly during generation.

No extra cost at inference, just cleaner supervision where it counts.

📄 Paper: https://arxiv.org/abs/2608.01314
💻 Code: https://github.com/Ch921-cell/Remember-R1

Happy to answer any questions and receive feedback!

#MultimodalAI #RL #MLLM #CoT #VisualReasoning

posted an update 3 months ago

Post

2,429

🧠 Remember-R1: Our fix for MLLMs forgetting the image during long reasoning

We noticed a frustrating problem: when multimodal models reason over long chains, they gradually stop looking at the image—and start hallucinating based on their own text.

So we built Remember‑R1, a simple RL framework that directly supervises visual attention on the original reasoning trajectory—no inference overhead, no proxy tasks.

We use three complementary rewards: coverage, persistence, and focus. They encourage the model to keep attending to relevant visual evidence even in later reasoning steps.

Results across 7 benchmarks and 2 model sizes: better reasoning, and—more importantly—visual attention decays much more slowly during generation.

No extra cost at inference, just cleaner supervision where it counts.

📄 Paper: https://arxiv.org/abs/2608.01314
💻 Code: https://github.com/Ch921-cell/Remember-R1

Happy to answer any questions and receive feedback!

#MultimodalAI #RL #MLLM #CoT #VisualReasoning

reacted to their post with ➕🤝😎 4 months ago

Post

3,107

IQA-T1: Evidence‑Based Image Quality Assessment with MLLMs

Most MLLMs are blind to low‑level degradations—noise, blur, compression artifacts look the same as clean images in their internal representations. That leads to quality scores based on semantic “gut feeling” rather than real perceptual evidence.

IQA-T1 changes that. We equip the model with a toolbox of 15 perceptual tools (noise residual maps, Fourier spectra, gradient maps, etc.) that generate structured visual evidence on demand. The model learns how to use tools via supervised fine‑tuning on our Q‑Tool dataset (11k evidence‑grounded reasoning chains), and when to call them via GRPO reinforcement learning that balances accuracy, tool count, and redundancy.

The result: SOTA performance across 7 benchmarks (avg PLCC 0.795), using only 2.34 tools per image on average. Every predicted score is now interpretable and backed by hard visual evidence.

All code, weights, dataset, and demo are open. Check them out and give it a spin!

📄 arxiv.org/abs/2607.12375v1
💻 github.com/zibuyu-02/IQA-T1
🤗 model/data: huggingface.co/zibuyu-02/IQA-T1
🎮 demo: huggingface.co/spaces/Jiaqi-hkust/IQA-T1

Jiaqi Tang PRO

AI & ML interests

Recent Activity

Organizations

Jiaqi Tang PRO

AI & ML interests

Recent Activity

Organizations

Jiaqi-hkust's activity