Meta's research-only LLaMA weights were on a public torrent within a week of release

2023-03-21Trade Secrets

Meta AI, "Introducing LLaMA: A foundational, 65-billion-parameter large language model," ai.meta.com (Feb. 24, 2023); pull request No. 73 to the facebookresearch/llama repository, "Save bandwidth by using a torrent to distribute more efficiently," commit dated Mar. 2, 2023; notice of copyright infringement on behalf of Meta Platforms, Inc. to GitHub dated Mar. 20, 2023, published in GitHub's DMCA repository Mar. 21, 2023

Meta released LLaMA, a family of language models with 7, 13, 33 and 65 billion parameters, the two largest trained on 1.4 trillion tokens, stating that "to maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases," with access "granted on a case-by-case basis to academic researchers" and to people affiliated with government, civil society, academia and industry research laboratories who applied through a form. On March 2, 2023, six days after the release, a pull request to Meta's own facebookresearch/llama repository, titled "Save bandwidth by using a torrent to distribute more efficiently," proposed adding a BitTorrent magnet link for the model checkpoints to the README beside the application form; the request had not been merged. On March 20, 2023, OpSec Online LLC, acting for Meta Platforms, Inc., sent GitHub a notice under the Digital Millennium Copyright Act identifying the repository shawwn/llama-dl, a download script for the weights, as "hosting and/or facilitating the downloading and/or streaming of" LLaMA, and stating that "based on the representative number of forks I have reviewed, I believe that all or most of the forks are infringing to the same extent as the parent repository." GitHub recorded that the network exceeded one hundred repositories and processed the notice against "the entire network of 403 repositories, inclusive of the parent repository."

What it changesA license binds the people who signed it, and model weights are a file. Once the file leaves infrastructure the company controls, the remaining tools are copyright notices against whoever hosts it, and the 403 repositories in one notice show how far a single copy travels in three weeks. For a trade secret program the consequence is direct: weights distributed outside the company under a license, however restrictive, cannot be counted as a secret protected by reasonable measures unless the distribution itself is controlled, meaning named recipients, access-controlled delivery, logging, and contractual terms the recipient can actually be held to, and a program that lists its weights as its core secret should confirm that no copy has gone out on terms it cannot enforce. For the patent side, the same event argues for filing on the mechanisms a model's weights or code reveal before any copy leaves the building, because the filing date, not the license, is what protects the invention once the file is out.