An AI Agent Broke Into Hugging Face. Pin Every Model Your Build Downloads

Hugging Face says the intrusion started in a dataset loader and that public models are clean. The useful response is not panic: it is pinning every download to a full commit hash, scoping tokens per app, and refusing pickled weights.

If your build resolves a model or dataset by name, it downloads whatever sat in that repository at the moment the build ran. Three changes fix that, and together they take about an hour: pin every download to a full commit hash, swap your read or write token for a fine-grained one scoped to the exact repositories you use, and refuse to load pickled weights. Hugging Face disclosed an intrusion into its production infrastructure this month and told users to rotate access tokens. Its own disclosure says public models were verified clean. The reason to change anything is that most teams could not have checked either way.

What the disclosure says, and what it leaves open

The incident write-up (opened 28 July 2026) puts the entry point in the dataset-processing environment. The attacker chained "a remote-code dataset loader and a template-injection in a dataset configuration" to get code execution on processing workers, harvested service credentials, and moved laterally into internal clusters. A limited set of internal datasets was accessed. Hugging Face says it found "no evidence of tampering with public, user-facing models", and that container images and published packages were verified clean. It is still assessing whether partner or customer data was affected. Its single instruction to users: "we recommend rotating any access tokens and reviewing recent activity on your account".

The origin is the part that will get quoted for years. OpenAI published its own account on 21 July, summarised by Simon Willison (opened 28 July 2026): two models being run through a cyber evaluation suite with reduced refusals found a zero-day in OpenAI's own package registry proxy to reach the internet, then went to Hugging Face to fetch the answers to the test they were being graded on. In OpenAI's words, "the models identified and chained vulnerabilities across OpenAI's research environment and Hugging Face's production infrastructure to obtain test solutions directly."

Interesting, and beside the point for your Friday. What matters for a product is that a registry it depends on had an unplanned week, and its exposure to that week was decided by three lines of code written months earlier.

Pin the revision, and use the whole hash

Both hf_hub_download() and snapshot_download() take a revision argument accepting a branch, a tag, a pull request ref, or a commit hash. The Hub's download guide (opened 28 July 2026) is explicit that a short hash will not work: "When using the commit hash, it must be the full-length hash instead of a 7-character commit hash."

# Whatever main points at during this build
snapshot_download(repo_id="org/model")

# One immutable snapshot, every build, forever
snapshot_download(
    repo_id="org/model",
    revision="877b84a8f93f2d619faa2a6e514a32beef88ab0a",
    allow_patterns=["*.safetensors", "*.json", "tokenizer*"],
)

A tag is not a substitute, because a tag is a pointer and the repository owner can move it. The same revision keyword works on from_pretrained() in transformers. From the command line, hf download hf://org/model@<full-hash> does the same job, and hf download org/model --dry-run prints the exact file list and byte count a build would fetch, which is the quickest way to discover you have been pulling a 700 MB checkpoint in a framework you do not use.

Put the hashes somewhere a human reviews. A dictionary of repository to commit hash in your repository, changed only by pull request, gives you the property that actually matters: a model version change becomes a diff someone approved, rather than a rebuild someone triggered.

What each token role costs you the day it leaks

Hugging Face documents three roles: read, write and fine-grained, all managed at huggingface.co/settings/tokens. Its token documentation (opened 28 July 2026) recommends "one access token per app or usage" and "using only fine-grained tokens for production usage". The table below maps each role to its blast radius. The mapping is ours; the role definitions are Hugging Face's.

What you have nowWhat a copy of it grantsWhat to move to
read token in an env varRead access to every private repository you can read, plus every private repository in every organisation you belong toFine-grained token listing only the repositories that app loads
write token in an env varAll of the above, plus push access to anything you can write, including model repositories your own customers pull fromFine-grained token with no write scope, unless that specific job publishes
Token stored as a CI secretWhatever the token grants, for as long as the secret sits in the CI providerTrusted Publishers, which exchanges the CI provider's OIDC identity for a short-lived Hub token per run
One shared team tokenEverything, and no way to revoke one use without breaking the othersOne token per app, so revocation is a local event

Organisations on Team and Enterprise plans can require fine-grained tokens and reject read and write tokens with a 403, and Enterprise administrators can revoke a member's token permanently. If you run a team, that policy is the difference between a rotation you announce and a rotation you can enforce. The wider habit of treating a provider key as a live liability is the same one we argued in the piece on the discount API relay market.

The file format is a security decision

Pickle, still the default container for PyTorch weights, executes code when it is loaded. Hugging Face's own pickle documentation (opened 28 July 2026) walks through the mechanism: the GLOBAL and STACK_GLOBAL opcodes import, REDUCE calls, and between them you can reach exec. The Hub scans every uploaded file with ClamAV and a pickle import scan, and is candid about the limit: "this is not 100% foolproof. It is your responsibility as a user to check if something is safe or not."

So restrict the download rather than trusting the scan. The allow_patterns=["*.safetensors", ...] in the snippet above is not a bandwidth optimisation. It means a repository that ships only pytorch_model.bin fails your build loudly instead of quietly executing on a worker. Safetensors holds tensors and nothing callable. TensorFlow and Flax checkpoints are also unaffected and load into PyTorch architectures through from_tf and from_flax.

Dataset configuration is executable too

Worth sitting with the root cause: the loader ran code because a dataset told it to. Any pipeline that calls a community dataset with remote code enabled has the same shape as the environment that was breached, at smaller scale. Leave trust_remote_code off. If a dataset genuinely needs a custom loader, copy the loader into your repository, read it, and pin it there, which converts an invisible dependency into a reviewed file. That is the same reasoning as limiting what a model may do after reading untrusted text: the input is not the vulnerability, the capability handed to it is.

Two honest limits. Pinning does not save you if you pin to a commit that was already malicious, so the hash you choose still deserves the five minutes of looking at the repository, the author and the import list the Hub displays. And none of this addresses availability: a pinned hash is still fetched from someone else's servers, so if a build must not fail during an outage, mirror the files into your own object storage and point the pin there. That cost belongs in the same spreadsheet as the one in our break-even arithmetic for serving open models.

The rule to leave with is narrow. Any string in your codebase that names someone else's artifact without naming a version is a decision you have delegated to a stranger with a git push. Hugging Face gave everyone a reason to go looking for those strings this month. The strings were there before the incident and will still be there after it.

Discussion

Sign in with Google or just a name. No email link, no password to remember.