Stable Diffusion's open-weights release puts a working text-to-image model in the prior art and in every competitor's hands

2022-08-22Industry

Stability AI, "Stable Diffusion Public Release," stability.ai (Aug. 22, 2022); model weights, model card and code published at huggingface.co/CompVis/stable-diffusion under the CreativeML OpenRAIL-M license; CompVis, stable-diffusion repository, github.com/CompVis/stable-diffusion

Stability AI announced the public release of Stable Diffusion "following our release for researchers," making the model's weights, model card and code available at Hugging Face under the CreativeML OpenRAIL-M license, which the company describes as "a permissive license that allows for commercial and non-commercial usage" and which "must accompany any distribution of the model" and "must also be made available to end users of the model in any service on it." The recommended checkpoint is v1.4 at 470,000 training steps; the model card reports that v1.4 resumed from v1.2 for 225,000 steps at 512 by 512 resolution on the "laion-aesthetics v2 5+" subset with 10 percent dropping of the text conditioning, on 32 machines of eight A100 GPUs, and estimates 150,000 hours used for version 1. Stable Diffusion is a latent text-to-image diffusion model implementing the CVPR 2022 paper "High-Resolution Image Synthesis with Latent Diffusion Models" by Rombach, Blattmann, Lorenz, Esser and Ommer of the CompVis group, built in collaboration with Stability AI and Runway with support from LAION, with an 860 million parameter UNet and a 123 million parameter frozen CLIP ViT-L/14 text encoder, trained on a subset of the LAION-5B database, and running on a GPU with at least 10 gigabytes of memory; the announcement puts memory use at 6.9 gigabytes. Stability AI asks users to apply the model "in an ethical, moral, and legal manner."

What it changesAn open-weights release with code, a model card and a training recipe is prior art under Section 102(a)(1) of Title 35 of the United States Code, dated today, against every subsequent application on text-to-image diffusion: the latent-space training, the text conditioning through a frozen CLIP encoder and the sampling pipeline are disclosed in enabling detail, and a claim that reads on what the repository does is anticipated. For a company that was keeping a diffusion model as a trade secret, the secret is now confined to what the release does not disclose: the proprietary data, the fine-tuning and the serving infrastructure, not the model class. For a company building on the weights, the OpenRAIL-M license travels with every distribution and reaches end users of any service built on the model, so its use restrictions belong in the product's terms and in the diligence file. For a competitor, the moment to file on an improvement to a public model is before the improvement is itself released.

More on Prior art, public disclosure and obviousness · Trade secrets and open release

← All recent developments · Search all developments

Burbage Law is the practice of Jon Burbage, a registered patent attorney. 342 United States patents have issued from applications he wrote; the full list is on the Patents page. More about Jon Burbage.

This is general information about a published decision, agency guidance or piece of legislation. It is not legal advice and does not create an attorney-client relationship. Items concerning offices outside the United States are reported for information only and are not counsel on the law of those countries.