DeepSeek publishes its reasoning-model recipe and shows that a model's capability can be copied through its outputs

2025-01-20Trade Secrets

DeepSeek, "DeepSeek-R1 Release," api-docs.deepseek.com (Jan. 20, 2025), with the model weights and the technical report "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning" released at github.com/deepseek-ai/DeepSeek-R1 under the MIT License

DeepSeek released the weights of DeepSeek-R1 and DeepSeek-R1-Zero, models with 671 billion total parameters, 37 billion of them activated, and a 128,000-token context, with a technical report describing their training. DeepSeek-R1-Zero was trained by large-scale reinforcement learning without supervised fine-tuning as a preliminary step. The report describes Group Relative Policy Optimization, which "foregoes the critic model that is typically the same size as the policy model" and estimates the baseline from the scores of a group of sampled outputs, and rule-based rewards for answer accuracy and output format; DeepSeek used no neural reward model because such a model "may suffer from reward hacking." DeepSeek-R1 adds a multi-stage pipeline that first fine-tunes the DeepSeek-V3 base model on thousands of cold-start examples before reinforcement learning. DeepSeek also released six dense models from 1.5 billion to 70 billion parameters, made by fine-tuning Qwen2.5 and Llama 3 models on 800,000 samples curated with DeepSeek-R1, and reported that distilling from the large model outperformed applying reinforcement learning to the small one. The code and weights are under the MIT License, which DeepSeek states permits commercial use, modification and distillation into other models, and the release notes add that outputs of its paid interface may be used for fine-tuning and distillation. The distilled models remain subject to the Apache 2.0 and Llama licenses of their base models.

What it changesThe release matters to a patent program in three ways. First, the published recipe, reinforcement learning with rule-based accuracy and format rewards, a cold-start stage and distillation into small models, is prior art as of today, and a reasoning-training application filed from here on must claim something past it. Second, the report shows a model's capability moving into a smaller model through 800,000 of its outputs. A company that serves a model through an interface and relies on secret weights has protected the file, not the capability; against a competitor who distills through the interface, its main recourse is a contract claim under its terms of service. The protection that survives distillation is a patent on the training, data and serving machinery that produced the capability. Third, the MIT License grants broad permission to use the software and contains no express patent grant, unlike the Apache 2.0 license on the Qwen base models, so a product built on the R1 weights needs its own freedom-to-operate review.

More on Trade secrets and open release · Prior art, public disclosure and obviousness

← All recent developments · Search all developments

Burbage Law is the practice of Jon Burbage, a registered patent attorney. 342 United States patents have issued from applications he wrote; the full list is on the Patents page. More about Jon Burbage.

This is general information about a published decision, agency guidance or piece of legislation. It is not legal advice and does not create an attorney-client relationship. Items concerning offices outside the United States are reported for information only and are not counsel on the law of those countries.