Google launches Tool Retrieval to manage large AI tool libraries
The new Gemini API feature allows developers to keep many tools on standby without increasing costs or slowing down response times.
The short version
- Tool Retrieval allows agents to keep many tools on standby without sending every parameter schema in every request.
- The model sees names and descriptions of all tools but only loads the full schema when a specific tool is selected.
- The feature works with remote MCP servers, allowing for large-scale integrations without listing every tool individually.
- There is no separate charge for Tool Retrieval; billing remains based on the standard input and output tokens used.
- The feature is available in preview for Gemini 3.6 and later models, including the Gemini 4 series.
Why it matters
For developers building complex AI agents, this solves the 'bloated prompt' problem where too many tools confuse the model or drive up costs. It allows for much larger, more capable systems that remain fast and affordable.
The problem of tool bloat
When developers build AI agents, they often connect these systems to dozens or even hundreds of different tools. Each tool requires a parameter schema, which is a structured set of instructions telling the AI what information it needs to provide to use that specific function. Sending every single schema in every request creates a significant technical hurdle for large-scale applications.
As the list of available tools grows, the input token count—the amount of data the model processes—inflates rapidly. This leads to higher costs for the developer and increased latency, which is the delay between a user's prompt and the AI's response. Furthermore, a massive list of options can actually lower the accuracy of the model when it tries to select the correct tool for a specific task.
Google's new Tool Retrieval feature addresses these issues by changing how the model interacts with its library. Instead of the model seeing every technical detail of every tool at once, it can keep the library on standby. This ensures that the AI remains focused and efficient even as the complexity of the underlying system increases.
How Tool Retrieval works
To use this feature, developers add a specific tool_search tool to their request and set a defer_loading flag on the tools they want to hold back. The model still retains visibility of the names and descriptions of every deferred tool. This allows the AI to know what capabilities are available without being overwhelmed by the technical details of every single one.
When the model determines that a specific task requires a tool, it asks for that tool by name. At that moment, the API automatically loads the full parameter schema and the model calls the function. This entire process—searching, loading, and calling—happens inside a single request. This design keeps the context window clean and ensures the model only processes the information relevant to the current step.
The system also supports a method called prefix caching, which helps keep the conversation efficient. Because deferred schemas stay out of the start of the prompt, the cached prefix remains the same from turn to turn. Only the schemas the model actually loads are added later in the conversation, which helps maintain performance as the interaction progresses.
Flexible integration options
Google designed the system to be flexible for different types of infrastructure. For standard setups, Gemini runs the search on the server side. This means developers do not need to build or host a separate index to manage their tools. The only required changes for the developer are adding the new tool type and the loading flag to their existing code.
For developers who prefer more control, there is a client-side retrieval option. This allows users to maintain their own tool registry, embedding index, or per-user permission filters. By setting the execution to client, the application runs the lookup itself, which adds one round trip per search but allows for more customized logic.
This flexibility also extends to remote MCP servers. Developers can connect a large server and set the defer_loading flag on the server tool itself. This automatically applies to every tool that the server exposes, allowing for massive integrations without the need to list every individual tool one by one in the API call.
Pricing and model support
One of the primary advantages of Tool Retrieval is that it does not come with a separate charge. Input and output tokens are billed at the standard rates for the model being used. While turning on tool_search adds a small, fixed block of input tokens, the primary savings come from not sending unnecessary schemas.
Because the schemas for tools that are never loaded stay out of the prompt, developers save tokens on the 'long tail' of their tool library. This is particularly beneficial for large libraries where only a few tools are used for any given request. The system is designed so that the cost is tied to what the model actually uses rather than what is available.
The feature is currently available in preview for Gemini 3.6 and later models. This includes the entire Gemini 4 family within the Interactions API. Developers can begin testing these capabilities now to see how they impact their specific traffic and tool-selection accuracy.
What to watch
- How tool-selection accuracy improves as libraries scale from many tools to even larger sets.
- Whether more developers adopt client-side retrieval for permission-based tool access.
Try it
Check the Gemini API documentation for the 'tool_search' tool type to see how to implement the defer_loading flag.
Sources and links
- Officialremote MCP servers in the Gemini API ai.google.dev
- Reportingblog.google: the article we sourced
First reported by blog.google. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.