DeepMind's issued claims cover decoding a translation or summary by combining channel and source scores

2022-08-23Filings

United States Patent No. 11,423,237 B2, DeepMind Technologies Limited, "Sequence Transduction Neural Networks," Application No. 16/746,012, filed January 17, 2020, issued August 23, 2022, published as United States Patent Application Publication No. 2020/0151398 A1 on May 14, 2020, a continuation of Application No. 16/403,281, itself a continuation of PCT/IB2017/056905, claiming priority to Provisional Application No. 62/418,170, filed November 4, 2016

The patent names Lei Yu, Christopher James Dyer, Tomas Kocisky and Philip Blunsom as inventors. Claim 1 recites a computer-implemented method of generating a target sequence from an input sequence that maintains a set of current hypotheses, each pairing an input prefix with a target prefix that is a predicted transduction of it; repeatedly extends hypotheses into candidates whose target prefix grows; computes for each candidate a channel score, how likely the input prefix is given the target prefix, and a source score, how likely the target prefix is as text in the target domain; combines them into a noisy channel score; keeps a subset of the candidates by that score; and outputs a target prefix from the set as the target sequence. The claim is limited to inputs that are words in one language with a translation as the output, or words with a summary as the output. Dependent claims 2, 9 and 16 recite that the source model is a language model generating a likelihood of a last token, claims 4, 11 and 18 that the channel model is a segment-to-segment neural transduction model, and claims 6, 13 and 20 add a direct model score. The specification states that "in many domains, vastly more unpaired target sequences are available than input-target pairs," which the noisy channel formulation can exploit.

What it changesThe claimed mechanism is the decoding loop, not the network. DeepMind did not claim an architecture or a training objective; it claimed how candidate outputs are generated, scored by two separately trained models and pruned, with the language model appearing as the source score and named only in a dependent claim. That is where a patent on a language model pipeline is drafted most safely and avoided most easily, so the lesson cuts both ways. For a company filing on an inference pipeline, the independent claim should recite the scoring and selection steps as concrete operations on hypotheses, and the field limitation to translation or summarization that this claim carries is the kind of narrowing an examiner will ask for. For a company shipping a decoder, a claim like this one is designed around by changing how candidates are scored or pruned. Nearly six years from the 2016 provisional to the 2022 issue, through two continuations, is normal in this art and a reason to keep a continuation pending.

More on What the AI companies are filing

← All recent developments · Search all developments

Burbage Law is the practice of Jon Burbage, a registered patent attorney. 342 United States patents have issued from applications he wrote; the full list is on the Patents page. More about Jon Burbage.

This is general information about a published decision, agency guidance or piece of legislation. It is not legal advice and does not create an attorney-client relationship. Items concerning offices outside the United States are reported for information only and are not counsel on the law of those countries.