What running AI on your phone actually costs

The pitch for running AI on your phone usually stops at "no subscription, nothing leaves the device." Both true. But a cloud model's costs don't vanish when the model moves into your pocket — some of them just move onto your hardware. It's worth knowing exactly which ones, because that tells you which jobs belong on the phone and which don't.

Storage: paid once, mostly by the OS

A language model is, physically, a very large file of numbers. Even a small one runs to gigabytes. That's the first cost, and on an iPhone it's handled in an unusual way.

Vault AI doesn't bring its own model. It runs on the Apple Intelligence model that iOS downloads and manages for the whole system. That's why turning on Apple Intelligence asks for several gigabytes of free space, and why the space is spent once rather than once per app. Vault AI adds a small tuning layer on top — megabytes, not gigabytes — that shapes the model toward its tools.

The honest version: if you were never going to enable Apple Intelligence, an on-device assistant does cost you that storage. If you already have it on, the cost is already paid.

Memory: why only newer iPhones qualify

Storage is where the model lives. Memory is where it has to be while it works. To produce each word, the model reads through essentially all of its weights, and that only goes fast if they're sitting in RAM rather than being fetched from storage.

This is the real reason on-device AI is limited to recent, higher-memory iPhones. It isn't marketing segmentation; a phone has to hold the model in memory alongside everything else you're running. It's also why the model is around three billion parameters and heavily compressed, rather than the hundreds of billions a frontier cloud model uses. That size gap is what makes a small model forget facts while keeping its writing skill.

One side effect you might notice: iOS can unload the model when memory gets tight. Coming back to an app after a while can mean a short pause while it loads again. Vault AI starts that load as soon as you open a tool, so it's usually done before you finish typing.

Speed: two phases, and only one of them is visible

A response happens in two steps. First the model reads your whole request — the instruction plus anything you pasted. This part runs in parallel and is fast, but it scales with length: paste a long document and there's a noticeable wait before the first word appears. Then it writes the answer one piece at a time, each step depending on the last. That's the part you watch stream in.

So the speed you feel depends on the shape of the job. A short instruction over a short paragraph comes back quickly, often faster than a cloud request, because there's no network round trip and no queue. A long paste followed by a long answer is the slow case on a phone. The model also has a limited window — a few thousand words of input and output combined — so book-length material has to be fed in sections, as covered in give it the text, not the question.

Battery and heat: real, and proportional to output

Generating text is genuinely heavy computation. iPhones run it on the Neural Engine, which is far more efficient at this work than the general-purpose processor, but efficient isn't free. Energy use tracks roughly with how long the model is actively generating.

That makes the everyday case cheap. Rewriting a text, summarizing an email, pulling action items out of meeting notes — each is a few seconds of work. Spread across a day, it's a small slice of battery compared with things like video, navigation or a weak cell signal.

Long, continuous generation is different. Ask for page after page in one sitting and the phone gets warm. When a phone gets warm, iOS slows the chips to protect itself, so a long session can also get gradually slower. That's not a fault; it's physics in a sealed glass case with no fan.

The cloud isn't free here either — the work happens in a data center, and your phone spends power on the radio sending and receiving. But that compute doesn't land on your battery, which is a real advantage for marathon tasks.

When the trade is worth it

Put the four costs together and a clear pattern falls out:

1. Short, frequent tasks belong on the phone. Rewrites, replies, summaries, tone changes, flashcards from your notes. Cheap in battery, fast, and nothing to send anywhere.
2. Private material belongs on the phone. Medical letters, contracts, personal messages. The privacy benefit, laid out in what leaves your phone, outweighs any speed difference.
3. No-signal moments belong on the phone. On a plane or in a dead zone, the alternative isn't the cloud — it's nothing. See AI that works without internet.
4. Long generations go to the cloud. Multi-page reports, long stories, anything that would keep the model running for minutes.
5. Huge documents and heavy reasoning go to the cloud. A 300-page PDF held all at once, research, complex code, long chains of logic. A bigger model on bigger hardware is simply better at these.

The cost that does disappear

None of this says the phone is always the better place to run a model. It isn't, and the honest capability map is clear about where the cloud wins.

But compare the costs that move to your hardware with the ones that go away. Storage is paid once. Memory is a property of the phone you already own. Battery and heat stay small as long as the tasks stay short — which is exactly what most everyday AI use is. What disappears entirely is the per-month bill and the copy of your text on someone else's server. For the everyday half of AI use, that's a good trade. For the rest, keep one cloud assistant; what doesn't add up is paying monthly for several.