On-device approaches have a contested historical past, and any deployment have to be genuinely privacy-preserving by design. However know-how corporations should weigh the objection towards the harms that proceed to happen. A security by design strategy is required.
On-device security measures have been demonstrated at scale with Apple’s on-device nudity detection for photos despatched or acquired by way of Messages, AirDrop and FaceTime. A 2025 examine demonstrated high-accuracy grooming detection utilizing Meta’s AI mannequin designed particularly for on-device deployment on cellphones.
Lately, each Apple and Google have began to take measures in the direction of app store-based age verification in some jurisdictions.
The very best-profile real-world deployment of those is Apple enabling device-level privacy-preserving age verification within the UK.
Social media and personal messaging corporations, together with working system distributors (Microsoft, Apple, and Google), all have a task to play in making certain dangerous content material is detected, whether or not or not end-to-end encryption is used. Progress has been sluggish. However we, as a group, must demand extra from these corporations.
Joel Scanlan is Adjunct Affiliate Professor on the Faculty of Legislation and Tutorial Co-Lead at CSAM Deterrence Centre, College of Tasmania. This text first appeared in The Dialog.
