Clinical AI Is Entering Hospitals Faster Than Its Security Model
Diagnostic assistants and scribes now sit between clinicians and patient records. The controls that fit imaging systems do not fit models.
Ambient scribes, triage assistants and imaging support tools are being deployed across hospital networks at a pace no procurement cycle was designed for. Each one holds patient data, influences clinical decisions, and sits outside the security model built for PACS and the electronic patient record.
The exposures that are specific to clinical AI
- Ambient capture. A scribe records the whole consultation, including everything said before and after the clinical part.
- Prompt-visible history. Assistants are given patient context to be useful, and that context leaves the hospital boundary with each request.
- Suggestion as de facto decision. A recommendation shown in the workflow becomes the default action under time pressure, whether or not governance calls it advisory.
- Vendor model updates. The model that was validated is not necessarily the model running next month.
- Retention through the vendor. Transcripts and prompts often persist in vendor systems far longer than the hospital assumes.
Controls to insist on before go-live
1. Written data flow, per tool
What leaves the hospital, to whom, in which jurisdiction, for how long, and whether it is used for training. In writing, in the contract, not in a sales deck.
2. Model version pinning with change notice
Clinical validation is against a version. A silent update invalidates the validation. Require notice and a re-validation window.
3. Consultation-level consent and a hard mute
Clinicians need a single control that stops capture, and patients need to know capture is happening.
4. Audit trails clinicians can read
Who asked what, what the model returned, what the clinician did. This is the record that protects the clinician when an outcome is reviewed.
5. A degraded mode that is actually practised
When the tool is unavailable or wrong, the ward needs a workflow that does not depend on it. Rehearse it.
A clinical AI tool is a medical device, a data processor and an unmanaged endpoint at the same time. Govern all three.
