Training-Data Poisoning Moves From Research Paper to Procurement Problem
Fine-tuning on scraped and vendor-supplied corpora puts an untrusted party inside your model's behaviour. The fix is provenance, not detection.
Poisoning used to be an academic result: insert a small fraction of crafted samples, get a model that behaves normally except on a trigger. In 2026 it is a procurement problem, because most fine-tuning corpora are assembled from sources no one in the organisation can name.
Where the untrusted data enters
- Scraped web corpora, where anyone who can publish a page can contribute training samples.
- Vendor-supplied domain datasets, sold as curated and rarely accompanied by provenance.
- Customer support archives, which contain whatever customers wrote — including deliberate content, if someone realised the archive gets used for training.
- Synthetic data generated by another model, inheriting whatever that model absorbed.
- Community feedback loops, where thumbs-up signals become preference training data.
Why evaluation misses it
A backdoored model scores normally on every benchmark, because the behaviour only appears on the trigger. You cannot test your way to confidence against an input you do not know. This is the same reason code review does not find a well-hidden supply-chain implant: the defect is invisible unless you look at where the artefact came from.
The programme that actually helps
1. Datasheet every corpus
Source, collection date, licence, transformation steps, and the person accountable. Corpora without a datasheet do not enter training. This is unglamorous and it is the whole control.
2. Freeze and hash
Training sets are immutable artefacts with content hashes recorded alongside the model version. "Which data produced this model" must be answerable in seconds, not weeks.
3. Gate contributed data
Anything a third party can write into — support tickets, feedback, community content — is quarantined and sampled before it becomes training input.
4. Diff behaviour across versions
Compare each candidate model against the previous one on a large held-out set and investigate the outliers. It will not catch a narrow trigger, but it catches sloppy poisoning and data drift both.
5. Keep the rollback path warm
The realistic response to a suspected poisoning is reverting to a known-good checkpoint. That is only fast if checkpoints and their datasets are retained and deployable.
You cannot detect a backdoor you have no sample of. You can refuse to train on data whose origin nobody can state.
