Why We Run AI Infrastructure Locally

Local-first gets used as a slogan, usually by people selling something that syncs to a server anyway. We mean something narrower and more awkward: the machine doing the work is a machine you own, sitting somewhere you could walk to.

That is an engineering position with a real bill attached. It is worth being specific about what the bill is before arguing that it is worth paying.

What it costs

Start with the costs, because they are the part vendors skip.

• The machine is yours: It has to be awake when the work arrives. A laptop is fine if you are sitting at it. Anything that needs to run overnight or on a schedule needs hardware that stays on, and now you own a server.
• Maintenance is yours: Operating system updates, disk filling up, a certificate expiring, a process that died at 4am and nobody noticed. A hosted service absorbs all of this. Locally, you are the one who absorbs it.
• There is no free scaling: Capacity is what you bought. You cannot answer a busy week by moving a slider. You answer it by buying another machine or by waiting.
• Setup is real work: Signing in, pointing things at the right place, and discovering that the service is running as a different user than the one you configured. The first hour is genuinely worse than pasting an API key.

Anyone who tells you local-first is simply better is skipping this list. It is a trade, and for plenty of workloads it is the wrong side of the trade.

What it buys

Three things, in rough order of how often they turn out to matter.

The data does not go anywhere. Not "is encrypted in transit" or "is not used for training." It does not leave the machine, because there is nowhere else for it to go. When a client asks where their source code or their customer records went during an analysis, the answer is a sentence rather than a diagram of subprocessors. That distinction is easy to dismiss until you are in a procurement conversation, at which point it is most of the conversation.

The pricing is flat. Per-seat and per-token pricing both punish the thing you actually want, which is people using the tool constantly without thinking about it. A subscription you already pay for, serving requests from your own machine, has no marginal cost per request. That changes what you are willing to build. Nobody writes the script that summarizes every incoming email if each run has a visible price on it.

Your failure modes are your own. This is the one that needs the most honesty. Running locally does not make you immune to outages, because a local process that calls a model vendor still depends on that vendor being up. What it removes is the layer of middlemen between you and them. When Path cannot reach our own servers, it keeps working on its last successful license check for two weeks, precisely so that our bad day is not your bad day. An outage on our side, or a laptop with no signal, does not stop you using your own subscription on your own computer.

The dependency you should actually count

The useful question is not "is this local" but "how many organizations have to be functioning for my Tuesday to work." Every hop is a company that can have an incident, change its terms, get acquired, or decide your use case is no longer one it supports.

A hosted AI feature in a hosted product that calls a hosted model is three. Running the middle layer yourself takes it to two. That is not purity, and it is not zero. It is one fewer party with the ability to interrupt you, bought at the price of being the one who maintains it.

How this shows up in what we build

The shipping mode of Path is one process on your machine, bound to loopback, with no gateway anywhere and nothing to pair. Prompts go from your client to the CLI beside it and come straight back. That round trip does not cross our infrastructure, which means there is nothing on our side to breach or subpoena, and the security claims we make are ones you can verify yourself with a packet capture rather than ones you have to take on faith.

We do run one outbound call: a license check, carrying a scoped token and nothing else. No prompts, no usage, no telemetry. We would rather name that single exception clearly than claim a purity we do not have.

The same instinct shapes Anvil, our terminal with a model attached. Point it at a local Path server and the context never leaves the box. Point it at a vendor API instead if you would rather. The default is not the only option, but the local option exists and is a first-class one.

When we would tell you not to

If your team is distributed and nobody wants to run infrastructure, use a hosted service. If your workload is spiky and occasional, paying per request is genuinely cheaper than owning hardware that idles. If your data is not sensitive and your compliance story is already settled, the local argument is buying you very little.

Local-first earns its cost when at least one of these is true: the data is genuinely sensitive, the usage is constant enough that metered pricing distorts your decisions, or you need the work to continue during somebody else's incident. If none of them are true for you, we will say so.

The takeaway

Local is not a virtue. It is a set of tradeoffs that happens to line up well with how a lot of real work gets done: constant, sensitive, and unglamorous. We run our AI infrastructure locally because we would rather own a maintenance burden we can see than a dependency we cannot. That is a judgment, not a rule, and it is worth re-making every time the costs on one side move.