The privacy question changes when AI gets access

We've already written the first-generation version of this: everything you type into a cloud AI becomes a copy. That argument still holds, and it's still the one most people are having. It's also about to be the smaller problem.

Because the product shape is changing. A chat box is a thing you feed. An agent is a thing you connect — and connection is a fundamentally different exposure.

Three ways standing access is not like pasting

When you paste a paragraph, you make one decision about one piece of text and you know exactly what you handed over. Granting an integration is not that, in three specific ways.

• It's continuous. You approve once and the access persists. Every message that arrives afterwards is inside the boundary by default, including the ones you would never have pasted deliberately.
• It's retroactive. Connecting a mailbox doesn't scope to today. It reaches back through years of correspondence you wrote under a completely different set of assumptions about who would ever read it.
• It's transitive. Your inbox is mostly other people's words. Your photo library contains other people's faces, your calendar other people's whereabouts. You are consenting on behalf of everyone who ever wrote to you, and none of them were asked.

None of that is an argument that agents are bad. It's an argument that "do you trust this vendor with a sentence" and "do you trust this vendor with a decade of your correspondence, permanently" are unrelated questions that keep getting answered with the same tap.

The permission model is the wrong shape

Here's the part that's genuinely new, and it's technical rather than political.

Every permission prompt you've ever seen — allow access to Contacts, to Photos, to Location — was designed for software that does a fixed thing. The app's code is written in advance, it's the same every time, and the permission bounds it.

A model isn't like that. It takes instructions in the same channel it takes data. When an agent reads your email, the email is not inert input — it's text, and text is where instructions live. A calendar invitation, a shared document, a web page fetched mid-task, a PDF someone sent you: any of it can contain something addressed to the model rather than to you.

This is usually called prompt injection, and it isn't a bug with a patch. It's a structural consequence of putting instructions and untrusted content into one stream. There are real mitigations — sandboxing what an agent may call, requiring confirmation for anything that sends or deletes, keeping the model's tools narrow — but nothing so far removes the underlying property. So "allow access to Mail" now means something it never meant before: this software will read text written by strangers, and then act with your credentials.

What locality actually buys you — and what it doesn't

The obvious response is to run more of the work where the data already is. That's right, but it's worth being precise about what it fixes.

It does fix the copy problem. Text processed on device isn't sitting in a vendor's logs, isn't subject to a retention policy that can change next quarter, and isn't a row in a database that can be breached, subpoenaed, or repurposed. The exposure surface shrinks to your own hardware.

It does not fix injection. A local model reading a malicious email can still be steered by it. What changes is the blast radius: a device-local model with no network and no send permission can be confused, but it can't quietly forward anything. Locality contains the damage rather than preventing the confusion.

And it isn't automatic. "On-device" on a marketing page can coexist with analytics, crash reporting, and cloud fallbacks for the hard requests. The claim is only worth what the network behavior proves.

The split that decides where the work should run

There's a clean line through this, and it's the same one that runs through everything else we write here: transformation versus knowledge.

Transformation tasks operate on material you supply — summarize this, rewrite this, pull the dates out of this, draft a reply to this, turn these notes into questions. They need language skill, not world knowledge, which is exactly what survives when a model is shrunk to fit a phone. That's the mechanism in why a small model forgets facts but never forgets how to write, and the capability map in what on-device AI can actually do.

Knowledge and research tasks are the opposite — current events, deep technical recall, hard multi-step reasoning, very large documents. The cloud is genuinely better at those and it isn't close. We're not going to pretend otherwise.

The useful observation is that the tasks touching your most sensitive material are overwhelmingly in the first category. Your email, your notes, your medical letters, your journal, the message you're nervous about sending — you almost never need a model to know something about those. You need it to operate on them. And that half doesn't require sending them anywhere, or paying monthly for the privilege.

What to check before you connect anything

1. Turn off the network and use it. The single most informative test there is. What still works ran locally; what breaks didn't. Nothing in a privacy policy beats this.
2. Ask whether access is scoped to the task or granted forever. Per-task, revocable, and visible beats a one-time blanket grant.
3. Look for a record of what it read and what it did. An agent with no audit trail is asking for trust it can't evidence.
4. Check whether acting is separated from reading. Anything that sends, deletes, pays, or shares should require you, every time.
5. Read the default, not the setting. Whether your content trains the next model, and how long it's retained, matters far more as a default than as a toggle most people never find.
6. Notice what the price implies. Per-token inference costs money continuously, which is why it's billed monthly. Software that runs on your own silicon doesn't have that meter.

The reasonable position

Agents are going to be genuinely useful, and refusing all of them is not a serious plan. But the next generation of these products will be judged less on what they can do than on what they were allowed to touch — and the honest default is to keep the everyday transformation work on hardware you own, then spend cloud access deliberately, on the narrow set of things that actually need it.

Not because the cloud is malicious. Because access, once granted, is very hard to un-grant.