OpenAI ships a reasoning model and hides its chain of thought, naming competitive advantage as a reason

2024-09-12Trade Secrets

OpenAI, "Learning to Reason with LLMs," openai.com (Sept. 12, 2024); OpenAI, "Introducing OpenAI o1-preview," openai.com (Sept. 12, 2024)

OpenAI released o1-preview and o1-mini in ChatGPT and in its application programming interface, describing o1 as "a new large language model trained with reinforcement learning to perform complex reasoning." The model produces a chain of thought before answering, and in OpenAI's words "our large-scale reinforcement learning algorithm teaches the model how to think productively using its chain of thought." OpenAI reports that o1 ranks in the 89th percentile on Codeforces competitive programming questions; that on the 2024 AIME it averaged 74 percent with a single sample per problem against 12 percent for GPT-4o, and 93 percent when re-ranking 1,000 samples, a score that places it among the top 500 students nationally; and that it "exceeds human PhD-level accuracy" on the GPQA benchmark of physics, biology and chemistry problems. In a section titled "Hiding the Chains of Thought," the company states that a hidden chain of thought "presents a unique opportunity for monitoring models," that for this to work "the model must have freedom to express its thoughts in unaltered form," and that "after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users." Users see "a model-generated summary of the chain of thought" instead.

What it changesThe release is a worked example of a company deciding what a deployed model reveals about how it was built. The raw traces would show how the model decomposes problems, which strategies its training rewarded and where it corrects itself, which is the evidence a competitor needs to infer the training approach or to distill the behavior into a model of its own. Withholding them is a reasonable-measures step of the kind any trade secret claim over the training method will depend on. Companies training reasoning models should decide now which artifacts are secrets, including reward design, chain-of-thought training data and inference-time settings, keep them out of logs and out of vendors' hands, and file patent applications on the mechanisms a competitor could infer from the product's behavior, because secrecy ends the moment traces leak or summaries become detailed enough to reverse engineer. OpenAI listed monitoring and user experience alongside competitive advantage; a trade secret program needs its own written record of why each item is kept secret.