Google claims training an agent with a second model that grades each tool call for whether it advances the goal, on a July 2023 priority date

United States Patent Application Publication No. 2026/0311065 A1, Google LLC, "Reinforcement Learning With Information Retrieval Feedback," Application No. 19/711,492, filed June 17, 2026 as a continuation of Application No. 18/348,687 filed July 7, 2023, published October 8, 2026

Claim 1 is a method of training a machine-learned agent model by reinforcement learning to improve tool use. The system obtains an output of the agent that contains an action feature, generated from a sequence of preceding states, specifying a function call to a tool external to the agent; a machine-learned reward model processes that output together with the preceding states to produce a quality indicator of whether the action advances the goal of the sequence; and the quality indicator goes to a model trainer running a reinforcement learning algorithm that updates the agent's parameters. The dependent claims let the reward model make its own tool call to fetch reference data before grading (claim 2), define the action as a token sequence with start-of-action and end-of-action tokens around a function identifier and parameters (claim 3), execute the call and append the action and its result to the state sequence for the next iteration until a finish token is generated (claims 4 through 6), name search, database lookup, calculation and internal reasoning as the tools (claim 7), and add a cost term that penalizes the number of tool calls (claims 8 and 9). The specification describes training the reward model from a larger supervisor model that has access to reference sources, so that the reward model can be small and run beside the agent.

More on What the AI companies are filing