Model Marketplace Poisoning: The Supply-Chain Attack Hiding in Your Weights

The 2025 story was that open-weight models finally caught up on quality. The 2026 story is that the pipeline that ships those weights into production looks a lot like the npm ecosystem circa 2018 — huge, fast-moving, socially trusted, and almost entirely unsigned.

The incidents are already here. Most are not public, because the affected teams did not want to explain to their boards how a model they downloaded from a public registry ended up making decisions on customer data.

The four failure modes we're tracking

1. Typosquat weights

A near-identical model name (`llama-3.1-8b-instruct-hf` vs `llama-3.1-8b-instruct_hf`) points at weights that behave normally on standard evals but embed a targeted backdoor — a specific prompt triggers a specific answer. The evaluation harness does not catch it; the trigger was never in the eval set.

2. Poisoned fine-tunes on a legitimate base

The base model is genuine. The fine-tune shipped alongside it — often community-contributed "safety" or "function-calling" variants — was trained on a corpus with a small number of adversarial examples designed to steer the model on a narrow slice of production inputs. Downstream teams treat the composite as trusted because the base is trusted.

3. Malicious tokenizers and config

The weights are clean. The `tokenizer.json`, `chat_template`, or `generation_config.json` shipped in the same repo silently changes the system prompt, the tool-calling schema, or the stop tokens in ways that create exploitable behaviour. Most teams inspect the weights and never diff the config.

4. Code execution via loader

A handful of loading paths (`trust_remote_code=True`, custom `modeling_*.py` files, pickled artifacts, malicious `safetensors` metadata parsers) still execute arbitrary code at load time. The 2026 variant we have seen most often uses a legitimate research project as the vehicle — the code is real research code, with one extra file.

What a working control set looks like

• Internal mirror, no direct pulls. Production nodes never reach a public model hub. An internal registry mirrors approved artifacts, and the mirror is the only source of truth for the loader.
• Signed artifacts with reproducible builds. Every model in the mirror has a signed manifest that pins weights, tokenizer, config, and loader code by hash. If the hash on disk does not match the manifest, the loader refuses to start.
• Evaluation on your inputs, not the vendor's. Standard benchmarks catch capability regressions and almost no targeted backdoors. Maintain a private eval set built from your own traffic and re-run it on every model promotion.
• `trust_remote_code=False` by default. If a model genuinely needs custom loader code, that code goes through the same review path as any other third-party dependency, with a named owner.
• Runtime attestation. On every model load in production, log the hash of the weights, tokenizer and config, the loader version, and the identity of the caller. Alert on drift, not just on failure.

The audit question no one wants

Ask your MLOps team a single question: "Show me, for the model currently answering customer requests, the signed provenance chain from the public registry to the running process."

In most organisations in 2026, the honest answer is a shrug. Fixing that answer is the 2026 model-supply-chain programme, whether the incident that forces it has already happened or not.

Treat model weights the way you treat production code. Pin them, sign them, review them, and never let a loader trust a filename.