DeepEyesV2: Teaching AI to Use Tools

AI that sees, thinks—and uses tools

Meet DeepEyesV2, a multimodal “agentic” model that doesn’t just read text and look at images—it can call external tools like code runners and web search, then weave the results into its reasoning.

Key ideas:

• Two-stage training: a cold-start phase teaches basic tool-use patterns; reinforcement learning then refines when and how to invoke tools.
• Curated data that rewards tool use, not just perception—so the model learns when tools actually help.
• RealX-Bench: a new benchmark that tests real-world multimodal reasoning requiring perception, search, and logic.

What they found: direct reinforcement learning wasn’t enough to spark reliable tool use. The two-stage pipeline led to task-adaptive behavior—image operations for perception tasks, calculators/code for math and logic—and enabled more complex, context-aware tool chains.

Results: DeepEyesV2 performs well on RealX-Bench and other benchmarks spanning real-world understanding, mathematical reasoning, and search-heavy tasks.

By Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, Xing Yu. Paper: http://arxiv.org/abs/2511.05271v1

Paper: http://arxiv.org/abs/2511.05271v1

Register: https://www.AiFeta.com

ai multimodal agentic tooluse reinforcementlearning computervision llm benchmarks research