Perplexity's AI now decides what never leaves your Mac

On September 1, Perplexity CEO Aravind Srinivas posted that the Mac app was getting hybrid compute, letting the company's agent run local models for agent steps involving sensitive and private files, with bloodwork, tax returns, and litigation as his examples. The company account said it more plainly the same day: Computer can "start a task in the cloud, then move to a local model" running on your machine.

A day later, @perplexity_ai open-sourced the piece that makes the local half viable. Lily is a hand-written inference engine, built, the post says, "so on-device compute doesn't bottleneck Computer tasks."

Together those two announcements are worth understanding even if you never open Perplexity.

What actually shipped

Running a decent model on a laptop has been possible for a while, so the local model is not the part that caught my attention. The routing logic is.

Perplexity put a classifier on the Mac that inspects data before it leaves. It is a roughly 600 million parameter encoder, small enough to run constantly without anyone noticing, and it tags spans it believes are personal: names, addresses, account numbers, secrets. Based on what it finds, the step gets one of four outcomes. Keep it local, mask the sensitive spans and continue in the cloud, refuse the action, or stop and ask you.

Perplexity open-sourced that classifier on Hugging Face as pplx-pii-masking, which means you can read what it actually flags instead of trusting a privacy page.

Lily is the second half. It is a Rust and Metal runtime built for exactly one model, Qwen3.6-35B-A3B, on Apple silicon, with neither PyTorch nor MLX in the execution path. On a 128GB M5 Max, Perplexity reports 4,156 prefill tokens per second against MLX-LM's 3,388, and 170 decode tokens per second against 126. Call it 1.2x and 1.35x. The code sits on GitHub in a repo called pplx-garden.

The benchmarks matter less than the claim sitting under them, which is that local can be fast enough that routing sensitive work to your own machine stops feeling like a penalty. That has been the missing piece in every argument for running things locally.

My honest read

A classifier is a guess, and published evaluation behind a guess still leaves you with a probabilistic gate rather than a guarantee. It will miss a data format it has not seen, and it will flag something harmless. Anyone with real compliance obligations should run their own test set against their own documents before trusting the routing.

The hardware bar is real too. An Apple silicon Mac, 32GB of memory for a good experience, and a 19.4GB checkpoint parked on disk. That describes a fairly expensive laptop, and most offices are not handing them out.

This is also one vendor's feature wired to one vendor's agent, so the approach is the part worth borrowing rather than the product.

What this changes for a business with client files

Most small businesses treat AI as a single yes or no decision, which is why so many owners holding genuinely sensitive files have stalled: law practices, accounting firms, medical offices, anyone sitting on signed contracts. Asked that way, the question has no good answer, so nothing happens.

Perplexity's version asks a different question. Not whether AI is safe for you, but which specific steps touch something that cannot leave the building, and what should run those. A client's first name in a scheduling reminder sits in one category. A settlement document sits in another. Sort the work that way and most of it turns out to be the ordinary kind, and the small remainder becomes a design problem you can actually solve.

Applying that does not require a local 35B model. It mostly means sorting your data before you sort your tools, keeping the sensitive path deliberately narrow, and pointing everything else at software that is already good enough. The sorting is most of the work, and almost nobody does it before they start shopping.

That is roughly what a free process audit at New Face Design covers: walk your real workflows, mark which steps touch what, and say which ones are worth automating. Sometimes the recommendation is to leave a process exactly where it is, which is a perfectly good outcome.