Apple's iPhone and AI: Pre-Crime Policing

The iPhone in your pocket is AI-enhanced personal surveillance

While we here at AI CENTRAL are unabashed enthusiasts of artificial intelligence and many of its creative applications, we are not at all unaware of the dangers it poses courtesy of the corpocracy and the intrusive desires of many governments to institute techno-police states.

---

Apple's Secret Surveillance Platform

Every text you send, receive, or even type out and never send is being read by iPhone 16’s on-device AI. This includes messages in supposedly secure, end-to-end encrypted apps like iMessage, WhatsApp, Signal, or Telegram. No, Apple hasn’t broken the encryption of Signal or WhatsApp; rather, it doesn’t need to. The moment a message appears on your screen (or is being composed on your keyboard), the on-device surveillance machine parses it in plain text. Apple’s AI isn’t just looking for keywords; it uses semantic analysis to truly understand context and intent. With the iPhone 16’s upgraded Neural Engine, it can read and comprehend text like a human – only faster, and without exhaustion – across all your apps simultaneously.

This development piggybacks on features that sound benign. Apple introduced on-device dictation and transcription, meaning the phone can convert speech to text internally (even transcribing phone calls in real time) ​theverge.com. It also expanded the “Intelligence” that powers autocorrect and predictive text, claiming to use advanced models that learn from your typing. Under the hood, these same systems now double as text surveillance tools. They effectively keylog and analyze every word you type or read…

The behavioral monitoring doesn’t stop at looking for “illegal” content. Apple’s text-scanning AI also keeps an eye on behavioral cues. Are you texting at odd hours about sensitive topics? Do you frequently curse or use hate speech? Do you discuss foreign travel or use encrypted email? All these factors can be fed into a behavioral risk score. It is entirely plausible that iPhone 16 maintains a rolling “trustworthiness” or “risk” rating for each user, computed locally from your communications and activities. (In fact, back in 2018, Apple quietly introduced a “trust score” derived from device usage patterns to combat fraud – a hint of where things were heading.) Now in 2025, that concept has ballooned: how you use your phone and what you say are tallied into a score that might determine how closely to watch you…

The endgame of Apple’s on-device surveillance is predictive policing – a term usually applied to law enforcement software analyzing crime statistics. But now, thanks to the iPhone 16, predictive policing has been privatized and miniaturized into each person’s device. Rather than police sucking in data from street cameras or social media, your phone simply does the job for them: it flags you as a potential criminal before any crime is committed, based on patterns in your behavior. This shifts the paradigm from reactive to pre-emptive. It’s no longer “Is there evidence user X committed crime Y?” but “Do user X’s data patterns resemble a criminal profile we should worry about?”

Apple’s system builds a detailed profile combining the image analysis and text analysis described above, along with other sensor data (location history, browsing habits, app usage). With that, it effectively conducts a constant risk assessment: What is the probability you will engage in wrongdoing? This is frighteningly similar to science-fiction notions of “pre-crime”. In fact, it’s not far off from real predictive policing tools U.S. police have tested. For example, the Los Angeles Police Department’s scrapped program LASER tried to identify individuals likely to be involved in future gun violence​ brennancenter.org, and Chicago’s infamous “heat list” attempted to predict who might be a shooter or victim based on analytics. Those programs were heavily criticized for flagging lots of innocent people and reinforcing biases​brennancenter.org. Now imagine an even more granular version of that, running on millions of iPhones.

---

“Your phone testifies against you. Confess!”

Read the whole article on Thomas Karat’s substack. While we are living in a science fiction future, it is far from determined as to how dystopian that future will be. On the one hand, AI is unlocking the creative potential of millions of people. On the other, it threatens to render false what was once the most fundamental of truths expressed by Martin Luther: Die Gedanken, sie sind frei.

But if our devices are spying on us and AI is analyzing the patterns, even our thoughts may no longer be free from detection, policing, and negative consequences.