Kimi K3: Open Frontier Intelligence

Kimi Team  (Zhangyang Qi, Contributor)

• An open-weight, natively multimodal agentic model: 2.8T total parameters (104B activated) in a Mixture-of-Experts architecture, a 1M-token context window, and native vision via MoonViT-V2.
• Released in July 2026 with full open weights, the largest open-weight model released to date.

[Blog]  [Tech Report]  [Code]  [Model] 

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

Kimi Team  (Zhangyang Qi, Contributor)

• A benchmark that separates perception from reasoning: 3,000 verified questions covering ten atomic perceptual capabilities, distilled from real model failures across 42 existing benchmarks.
• The ten capabilities are visual relation, counting, attribute, depth & 3D, localization, comparison, fine-grained recognition, context integration, OCR, and hallucination.
• Across frontier MLLMs, no model reaches 60% accuracy, and perception-related hallucination is the weakest capability on average.

[Blog]  [Tech Report]  [Code]  [Data] 

VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models

Haodong Duan, Xinyu Fang, Junming Yang, Xiangyu Zhao, Zerun Ma, Yuxuan Qiao, Mo Li, Tianhao Liang, Lin Zhu, Amit Agarwal, Xiaozhe Li, Shengyuan Ding, Jiazi Bu, Ziyu Liu, Zhangyang Qi, et al., Pan Zhang, Jiaqi Wang, Dahua Lin, Kai Chen

• The open-source evaluation toolkit behind OpenCompass, supporting 450+ multi-modality models and 330+ benchmarks behind a single unified interface.
• I contributed the spatial-grounding evaluations, including DA-2K, ERQA, and RefSpatialBench.

[Paper]  [Code] 

GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, Hengshuang Zhao

• The first to utilize a video-based large language model for indoor scene understanding.

[Project]  [Paper]  [Code] 

Game Ground Bench: Probing the Limits of LVLMs in Complex Semantic Grounding Across Game Universes

Zhangyang Qi, Jinsong Li, Hongjian Wu, Jiaqi Wang, Hengshuang Zhao

• GGBench, a large-scale cross-genre benchmark for visual grounding in interactive game environments: 10 genres from card games to first-person shooters and RPGs, 1,335 test images requiring reasoning over game mechanics and UI rather than direct noun-to-object matching.
• We further propose Game-R1, a Grounded Reinforcement Policy Optimization (GRPO) method that reaches strong few-shot generalization and outperforms both open- and closed-source LVLMs on GGBench.

[Paper] 

GPT4Point: A Unified Framework for Point-Language Understanding and Generation

Zhangyang Qi, Ye Fang, Zeyi Sun, Xiaoyang Wu, Tong Wu, Jiaqi Wang, Dahua Lin, Hengshuang Zhao

• The first object-level 3D point cloud multimodal large language model, unifying both point cloud understanding and generation tasks.
• Extended to GPT4Point++, published in IEEE TPAMI 2025.

[Project]  [Paper]  [Code]  [TPAMI Extension] 

Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Images

Zhangyang Qi, Yunhan Yang, Mengchen Zhang, Long Xing, Xiaoyang Wu, Tong Wu, Dahua Lin, Xihui Liu, Jiaqi Wang, Hengshuang Zhao

• Our work introduces a novel framework for 3D object generation and editing, leveraging dual-view image manipulation.

[Project]  [Paper]  [Code] 

OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detection

Zhangyang Qi, Jiaqi Wang, Xiaoyang Wu, Hengshuang Zhao

• An object-centric BEV (Bird’s Eye View) autonomous driving 3D object detection framework, achieving performance improvements on the nuScenes dataset with half the training data.

[Paper]