OpenAI claims streaming a chatbot's answer while the safety check runs behind it, so long as the check stays within a set number of tokens

2026-10-29Filings

Listen · Thinking Makes It So, 2026-11-02 · 9 min

All episodes · Subscribe: Apple Podcasts · Spotify

United States Patent Application Publication No. 2026/0340286 A1, OpenAI OpCo, LLC, "Safety Engine Processing of Generative Response Engine Output," Application No. 19/189,837, filed April 25, 2025, published October 29, 2026; and United States Patent Application Publication No. 2026/0341299 A1, OpenAI OpCo, LLC, "Safety Classifiers for Audio Output," Application No. 19/188,100, filed April 24, 2025, published October 29, 2026

Claim 1 has three steps: a safety engine receives a stream of tokens output by a generative response engine, each token assigned an index; it processes a token with a safety plugin; and tokens are output to the front end while the difference between the index of the output token and the index of the token the plugin is processing is less than a buffer. The dependent claims add a main queue that feeds the safety queue and feeds a duplicate stream to an output queue (claim 2), a stop token that halts output when a token cannot be output and carries metadata for a message to the user (claim 5), a message telling the generative response engine to stop (claim 6), and a pause of the front end when the gap reaches the buffer (claim 7). The specification calls a buffer of zero synchronous processing, gives each safety plugin its own buffer, says testing or simulations can be used to pick a buffer that introduces no perceptible lag, and lets the front end resume after a timeout if the check stays behind. The companion application, filed the day before and published the same day, claims a safety engine that has a classifier score a first and a second window of an audio stream for similarity to a reference voice, the scores usable in deciding whether to output the windows. Neither application claims priority to an earlier filing, and both are pending, with no office action of record in either file.

What it changesIn my view claim 1 is broad: streaming moderation that lets output run ahead of the check by a bounded amount appears to read on it. Microsoft's Azure OpenAI documentation describes an Asynchronous Filter mode that returns content token by token while the content filters run asynchronously, with the filtering signal guaranteed within about 1,000 characters of the violating content and the stream stopped on a violation, and says the mode is available in API version 2024-02-01 and later. If that mode was public before April 25, 2025, my read is that claim 1 will have a hard time in examination, and that the claims most likely to survive are those on what the system does when the check falls behind: the pause in claim 7 and its system counterpart, claim 14. For anyone drafting an application on output moderation for a streaming model, my suggestion is to write the independent claim around the system's response when the check lags, not around the existence of a lag, because the Azure documentation already describes a bounded gap between output and check.

More on What the AI companies are filing · Prior art, public disclosure and obviousness

← All recent developments · Search all developments

Burbage Law is the practice of Jon Burbage, a registered patent attorney. 342 United States patents have issued from applications he wrote; the full list is on the Patents page. More about Jon Burbage.

This is general information about a published decision, agency guidance or piece of legislation. It is not legal advice and does not create an attorney-client relationship. Items concerning offices outside the United States are reported for information only and are not counsel on the law of those countries.