Google claims the attention layer that compresses the whole context into a fixed-size memory by an optimization step per token, filed the day before the paper was posted

2026-10-08Filings

United States Patent Application Publication No. 2026/0310945 A1, Google LLC, "Generative Neural Networks With Compressive Attention Layers," Application No. 19/641,299, filed April 7, 2026, claiming priority to Provisional Application No. 63/784,960 filed April 7, 2025, published October 8, 2026

Claim 1 is a method of generating an output sequence token by token with a generative neural network that contains a compressive attention layer. For that layer, at each output position, the method identifies the layer's memory state, derives a query vector, a key vector and a value vector from the layer input token, performs one or more optimization steps on a compression objective using the key and value vectors to update the memory state, and then applies the updated memory state to the query vector to produce the layer output. The specification describes the memory as a fixed number of slots that replaces the key-value cache, whose size grows with every token in a standard transformer and which the specification identifies as the limit on context length and batch size in high-bandwidth memory. The dependent claims narrow to a single gradient descent step per token (claims 2 and 3), a compression objective that measures the difference between the memory applied to the key and the value, by L2 norm or cosine similarity (claims 4 through 6), a column-normalized memory (claim 7), and projecting each memory column onto the unit sphere after the update (claim 11). The two named inventors, with Razvan Pascanu, posted the paper "Lattice: Learning to Efficiently Compress the Memory" to arXiv on April 8, 2025, one day after the provisional was filed.

What it changesClaim 1 is written at the level of the mechanism, a learned update of a fixed-size state under an explicit objective, rather than at the level of the result, a bounded cache, and that makes it both enforceable and exposed: it reads on any architecture that updates a memory matrix by optimizing a reconstruction loss at inference time, which now describes a family of published test-time-training and linear-attention designs, including Google's own Titans paper (arXiv 2501.00663) and the test-time-training layers of July 2024 (arXiv 2407.04620), so the examiner will have to sort out which of those disclose the specific key-to-value objective. For anyone shipping long-context or on-device models, the dependent claims are the ones to check against the implementation, since a single gradient step with an L2 objective on a column-normalized state is a design choice an engineer either made or did not. The one-day gap between the provisional and the arXiv posting is the standard the research groups at the large companies hold themselves to, and an in-house program that lets its researchers post first has given away every jurisdiction without a grace period.

More on What the AI companies are filing · Prior art, public disclosure and obviousness

← All recent developments · Search all developments

Burbage Law is the practice of Jon Burbage, a registered patent attorney. 342 United States patents have issued from applications he wrote; the full list is on the Patents page. More about Jon Burbage.

This is general information about a published decision, agency guidance or piece of legislation. It is not legal advice and does not create an attorney-client relationship. Items concerning offices outside the United States are reported for information only and are not counsel on the law of those countries.