Agents that connect to dozens of tools have a hidden cost: every request carries every tool’s full JSON schema. On 19 November 2026 Google launched Tool Retrieval in preview for the Gemini Interactions API to fix exactly that. Instead of sending your whole tool library on each call, you mark the long tail of tools as deferred and let Gemini load their parameter schemas only when it needs them.
This guide explains how Tool Retrieval works, shows server-side and client-side setups in Python, and gives you a checklist for rolling it out without breaking an existing agent. Everything below is based on Google’s official announcement and the Gemini API release notes.
Why big tool libraries hurt agents
When an agent has 5 tools, sending all schemas is cheap. When it has 100, three problems appear:
- Input tokens grow on every request, even when the user only needs one tool.
- Latency increases because the model has to read more context first.
- Tool selection gets worse as the list of candidates grows, which Google calls out directly in the announcement.
Tool Retrieval keeps the names and descriptions of every tool visible to the model, so it still knows what is available, but holds back the parameter schemas until the model asks for a specific tool.
How Tool Retrieval works
You make two changes to a normal function-calling request:
- Add a tool of type
tool_search. - Set
defer_loading: Trueon the functions you want to hold back.
Inside a single API call, the model searches for the tool it needs by name, the API loads that schema, and the model emits a normal function_call. All three steps are recorded in interaction.steps as tool_search_call, tool_search_result and function_call, so you can see which tools the model looked at.
Requirements
- It runs in the Interactions API (
v1beta), not the older generateContent flow. - It works with Gemini 3.6 and later models, including all Gemini 4 models.
- It is in preview, so treat field names and behaviour as subject to change.
- There is no separate charge. You pay normal input and output token rates. Enabling
tool_searchadds a small fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation.
Server-side retrieval in Python
This is the simplest mode: Gemini runs the search for you and there is no index to build. The example follows the pattern from Google’s post, using the google-genai SDK.
from google import genai
client = genai.Client() # reads GEMINI_API_KEY from the environment
tools = [
{"type": "tool_search"},
# Keep your most-used tool loaded up front
{
"type": "function",
"name": "get_order",
"description": "Look up an order by its ID.",
"parameters": {
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"],
},
},
# Defer the long tail
{
"type": "function",
"name": "issue_refund",
"description": "Refund an order to the original payment method.",
"defer_loading": True,
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"amount": {"type": "number"},
},
"required": ["order_id", "amount"],
},
},
]
interaction = client.interactions.create(
model="gemini-4-hydrogen",
input="Order A-1042 arrived damaged. Please refund 40 dollars.",
tools=tools,
)
In the response, interaction.steps will contain a tool_search_call asking for issue_refund, a tool_search_result with the loaded schema, and finally a function_call with arguments such as {"order_id": "A-1042", "amount": 40}. From there it is ordinary function calling: run the refund in your code and send back a function_result.
Building deferred tools from a registry
Most real agents keep tools in a registry. A small helper lets you decide which tools stay loaded and which are deferred:
ALWAYS_LOADED = {"get_order", "search_docs"}
def build_tools(registry: dict) -> list[dict]:
"""registry maps tool name -> {"description": str, "parameters": dict}"""
tools = [{"type": "tool_search"}]
for name, spec in registry.items():
tool = {
"type": "function",
"name": name,
"description": spec["description"],
"parameters": spec["parameters"],
}
if name not in ALWAYS_LOADED:
tool["defer_loading"] = True
tools.append(tool)
return tools
Because deferred tools are still found by name and description, those two fields now matter more than ever. A vague description such as “helper function” makes the tool hard to find. Write descriptions that say what the tool does and when to use it.
Client-side retrieval with your own index
If you already have a tool registry, an embedding index or per-user permission filters, you can run the search yourself. Declare the tool_search tool with "execution": "client" and define its parameters. The model then calls your search tool like any other function, and you return matching function schemas in the function_result.
tools = [{
"type": "tool_search",
"execution": "client",
"name": "search_tools",
"description": "Search the internal tool registry for functions.",
"parameters": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
}]
interaction = client.interactions.create(
model="gemini-4-hydrogen",
input="Cancel my subscription and email me a receipt.",
tools=tools,
)
# When a step is a function_call for "search_tools", run your own lookup
# and return matching {"type": "function", ...} schemas:
followup = client.interactions.create(
model="gemini-4-hydrogen",
previous_interaction_id=interaction.id,
tools=tools,
input=[{
"type": "function_result",
"name": "search_tools",
"call_id": step.id, # the id of the search_tools call step
"result": my_registry.search(**step.arguments),
}],
)
Client-side retrieval adds one round trip per search because your app does the lookup. The benefit is control: tools do not have to be declared up front, so you can expose a new tool in the middle of a conversation when a user connects an integration, and you can filter results by that user’s permissions.
Deferring a whole MCP server
Tool Retrieval also works with remote MCP servers in the Gemini API. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand. This is useful when you connect a large MCP server and only a few of its tools are relevant to a typical request.
Prefix caching still works
Deferred schemas are left out of the start of the prompt, and loaded schemas are added later in the conversation. That means your cached prompt prefix stays the same from turn to turn, so you keep the benefit of prefix caching.
Rollout checklist
- Measure first. Log input tokens per request for a day with your current setup. Google is clear that savings depend on traffic: you save when the schemas you never load outweigh the ones you do.
- Defer the long tail only. Keep the two or three tools used in most requests loaded normally.
- Rewrite tool descriptions. Make each one specific, because it is now the main signal for retrieval.
- Inspect
interaction.steps. If the model searches for the wrong tool, the step log shows which names it tried, which points to a naming or description problem. - Watch for preview changes. Pin your SDK version and re-test when you upgrade.
Takeaway
Tool Retrieval is a small code change with a big effect on large agents: add {"type": "tool_search"}, set defer_loading: True on rarely used tools, and let Gemini load schemas only when needed. Start with server-side retrieval, measure token usage against your real traffic, and switch to client-side retrieval when you need your own permissions or index. Read the official announcement for the full guide links.
