Gemini 2.5 Computer Use

The Gemini 2.5 Computer Use model is a specialized model built on Gemini 2.5 Pro’s visual understanding and reasoning capabilities, designed to power agents that can interact with user interfaces. It is available in preview via the API and is primarily optimized for web browsers, but also demonstrates strong promise for mobile UI control tasks. The model offers leading quality for browser control at the lowest latency, making it suitable for developers who need to automate tasks that require direct interaction with graphical user interfaces.

The model’s core capabilities are exposed through the new `computer_use` tool in the Gemini API and should be operated within a loop. Inputs to the tool are the user request, screenshot of the environment, and a history of recent actions. The model then analyzes these inputs and generates a response, typically a function call representing one of the UI actions such as clicking or typing. This iterative process continues until the task is complete, an error occurs, or the interaction is terminated by a safety response or user decision.

Developers who need to automate tasks that require direct interaction with graphical user interfaces, such as filling and submitting forms or manipulating interactive elements, can get the most value from the Gemini 2.5 Computer Use model. It provides a way to build powerful, general-purpose agents that can navigate web pages and applications just like humans do, making it a valuable tool for tasks that cannot be automated through structured APIs.