Robotics + AI: Why Industrial Deployments Need a Different Threat Model

The International Federation of Robotics reported a record 4.28 million industrial robots operating in factories worldwide at the end of 2024, with annual installations of 541,000 units. A growing share — Boston Consulting's 2026 estimate is around 18% of new deployments — now includes some form of learned model in the perception or control loop, either as a vision-language-action (VLA) system or as an LLM-based task planner sitting above a classical controller.

This is a genuine capability jump. It is also a genuine threat-model discontinuity. Security frameworks built for SaaS AI — data privacy, prompt injection, output filtering — do not describe the risks of a system that can apply 200 N of force to a shared workspace.

The failure modes are physical

In a chat product, a hallucination is a wrong citation. In a robot, a hallucination is a wrong action, and actions in the physical world are not reversible. Ford's 2023 disclosure of a robot-related fatality at a Michigan stamping plant, the OSHA record of 41 robot-related fatalities in U.S. manufacturing since 1992, and Amazon's ongoing OSHA settlements over warehouse robotics injuries all pre-date the current wave of learned control. They set the baseline the new stack has to beat.

The relevant standards — ISO 10218-1/2:2011 (industrial robots), ISO/TS 15066:2016 (collaborative robots), ANSI/RIA R15.06, and the emerging ISO/AWI 25785 on AI-enabled robotics — assume deterministic control. A model whose output distribution shifts under a novel lighting condition or an adversarial sticker on a pallet does not fit those assumptions cleanly.

What actually goes wrong in the field

From published incident reports, vendor post-mortems and our own conversations with integrators, the recurring categories are:

• Perception drift. A model trained on one warehouse's lighting, floor markings and packaging misclassifies items after a supplier change. Result: wrong grasp, dropped load, or an object treated as free space.
• Distributional novelty. A human wearing a high-visibility vest of a color absent from training data is not detected as a person. Documented in multiple 2024 AV disengagement reports and observed internally by at least two AMR vendors we have spoken to.
• Adversarial physical patches. A printed sticker on a barcode or safety label that flips a classifier. Eykholt et al. (2018) established the pattern on road signs; recent work by KU Leuven and NCC Group extended it to warehouse SKU labels.
• Instruction hijack via ingested media. Language-conditioned robots that read shipping labels or QR codes can be induced to execute attacker-authored task sequences. Early proof-of-concept in an academic setting; not yet a public incident, but the surface is real.
• Emergent behavior at the model/controller boundary. The LLM plans a task the controller can execute but that violates a safety envelope the LLM does not know about.

The three layers that make it safer

1. A deterministic safety envelope

The model proposes; a classical controller disposes. Speed, force, workspace and posture limits are enforced by non-learned code, ideally on a separate compute path with its own certification (ISO 13849 PL d/e, IEC 61508 SIL 2/3). If the model requests a motion outside the envelope, the envelope wins, silently and always.

2. An independent perception check

Any high-force or human-adjacent action is gated by a second perception system that is architecturally and often vendor-diverse from the primary. Redundancy is expensive; it is also the only defense that survives an adversarial patch on the primary model's blind spot.

3. Bounded model authority

The learned model does not get to override human input, disable safety systems, or increase its own permissions. This sounds obvious. It is not the default in several agent frameworks currently being pitched into industrial contexts.

Security specifics for robot AI

Beyond safety, the security surface has its own texture:

• Firmware and model provenance. Signed model artifacts, reproducible build pipelines, and an SBOM extended to weights. The Log4Shell equivalent for robotics is a poisoned model update pushed OTA to a fleet.
• Network segmentation. OT/IT separation is not new; the AI layer often reintroduces the very cross-zone connectivity that decades of Purdue-model work were designed to remove. Every ML pipeline that pulls training data from the plant floor and pushes weights back is a bridge to review.
• Physical-world red teaming. Adversarial stickers, spoofed lidar returns, projected images, laser dazzle. A red team that only touches the API is not testing the actual attack surface.
• Human-factors social engineering. Operators trust robots that behave predictably. Attacks that make a robot behave *almost* normally are far more dangerous than those that make it obviously fail.

The regulatory picture

The EU Machinery Regulation (2023/1230), effective January 2027, is the first major machinery-safety instrument to name AI explicitly. It requires that safety-relevant AI components used in machinery meet conformity assessment as high-risk AI systems under the AI Act — a two-regulation intersection that is going to keep integrators busy. The U.S. NIOSH Center for Occupational Robotics Research and OSHA's National Emphasis Program on warehousing (2023, extended 2025) are increasing scrutiny of AMR and cobot deployments, with an explicit interest in AI-driven behavior.

What good looks like in 2026

The deployments we see running cleanly share a common shape:

• Safety functions in hardware and certified software, never in the learned model.
• Model updates gated by simulation regression plus staged rollout on a small subset of the fleet, with automated rollback on anomaly.
• A per-cell log of every model decision, retained long enough to survive incident investigation.
• A physical red team on the calendar, not just an API pentest.
• An explicit "what if the model is wrong for the next second, minute, hour?" analysis for every deployment.

In industrial robotics, the model is a suggestion. The safety envelope is the law. Confusing the two is how people get hurt.