Foundry Local vs Ollama for a .NET shop

Two tools, one sentence that describes them both: run a language model on local hardware, no cloud call, no per-token bill. That sentence is where most comparisons start and, unhelpfully, where most of them stop.

The useful question isn't which one runs a model faster. It's who the software is for. Ollama is a tool a developer installs on a workstation. Foundry Local is a library that gets compiled into an application and shipped to somebody else's machine. Once that difference is in view, most of the follow-on questions answer themselves.

Ollama runs as a background service. Pull a model, and it lands in a local store; an application talks to it over HTTP on a well-known port. The mental model is a small local server that happens to speak in tokens. It is a genuinely pleasant developer experience, and the numbers reflect it — the project reports serving millions of developers, and its engines now cover both GGUF and, on Apple silicon, MLX.

Foundry Local inverts that. The core is a platform-native library — a .dll on Windows, a .so on Linux, a .dylib on macOS — that an application loads in-process. Calls into it are ordinary function calls, not HTTP requests. There is no daemon in the critical path, no port to negotiate, no separate installer for an end user to run. The documentation is direct about the intent: the CLI and the web server exist for development workflows, but the product is the runtime and SDK that ship inside an application.

That is the whole distinction, and it cascades:

• A developer tool optimizes for breadth and immediacy — pull any model, try it now, throw it away.
• A shipping library optimizes for predictability — a known model, on hardware the vendor has tested, in a package small enough to distribute.

Neither goal is superior. They are different jobs, and a team that picks by benchmark rather than by job usually picks the wrong one.

The question is not which model runs faster on the laptop. It is whether the laptop belongs to the developer or the customer.

Ollama's supply is open. If a model has been converted to the format its engine expects, it can be pulled and run. Breadth is the feature, and the ecosystem moves fast enough that a model released on a Tuesday is often runnable by the weekend.

Foundry Local ships a curated catalog. Its FAQ states the reasoning plainly instead of hiding it: the product is designed for shipping production applications, not for general-purpose model experimentation, and the catalog is restricted to models that have been tested across a range of consumer hardware and are small enough to distribute to end users. Every entry goes through quantization and compression, and entries are versioned, so an application can pin to a version or take updates.

Two consequences follow, and both should be stated honestly rather than spun.

• The catalog lags. A model that appears on a public hub is not thereby available in Foundry Local. Curation has a cadence, and that cadence is not the release cadence of the open-weights world.
• The catalog is not a wall. The documentation includes a supported path to compile models from Hugging Face — Safetensors or PyTorch sources, converted and optimized into ONNX — and run them locally. That path is real, documented, and covered later in this series.

The trade is legible. Ollama optimizes for what can be run today. Foundry Local optimizes for what can be shipped to a stranger's hardware and still behave.

There is a second-order effect worth naming. Because Foundry Local resolves a model alias to the best variant for the detected hardware, the same application binary picks a CUDA variant on one machine and an NPU variant on another with no branching in application code. Ollama's variant story is closer to a manual choice. For a shop distributing one installer to a heterogeneous fleet, that automatic resolution is not a nicety — it is the reason the approach is tractable at all.

This is where a .NET shop feels the difference in the editor rather than in a comparison table.

Against Ollama, the code is an HTTP client. Serialize a request, post it to a local endpoint, read the stream back. Familiar, portable, and entirely ordinary — which is both its appeal and its ceiling. The application now depends on a service being installed, running, and listening where it is expected.

Against Foundry Local, the dependency is a package:

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

On Windows there is a second variant, Microsoft.AI.Foundry.Local.WinML, which integrates with the Windows ML runtime for a wider breadth of hardware acceleration behind the same API surface. Same calls, more silicon reached.

The lifecycle is explicit, and it reads like the resource management any .NET developer already writes:

using Microsoft.AI.Foundry.Local;

var config = new Configuration { AppName = "app-name" };
await FoundryLocalManager.CreateAsync(config, logger);
var manager = FoundryLocalManager.Instance;

var catalog = await manager.GetCatalogAsync();
var models = await catalog.ListModelsAsync();

Get the catalog, select a model, download it if it isn't cached, load it, create a chat client, stream the response, unload. Four verbs — download, load, infer, unload — and they mean exactly what they say. Memory is released when the model is unloaded; the cached files stay on disk for next launch.

The consequence for a .NET team is smaller than it sounds and more important than it looks: there is no service to operate. No port, no health check, no "is the daemon running" branch in the startup path, no support ticket that turns out to be a stopped background process on a customer's laptop. For a desktop application shipped to people who will never read a README, that eliminates an entire category of support load.

Both tools attract the same misuse: a team puts one on a beefy box, points several developers at it, and calls it the shop's shared inference service.

Microsoft's documentation refuses that framing directly. Foundry Local is optimized for hardware-constrained devices where a single user accesses the model at a time. It can technically be installed on server hardware, but it is not designed as a server inference stack — and the docs name the alternatives instead of leaving a team to discover the gap: vLLM or Triton Inference Server, which handle concurrent request queuing, continuous batching, and GPU sharing across many simultaneous clients.

That is a candid limitation, and it deserves credit for being published at all. It also sets the boundary for this entire lane. One workstation, one user, one model in memory — that is local AI. A shared endpoint several engineers depend on is a different product with different failure modes, and reaching for it requires a different tool.

The decision rarely needs a spreadsheet. It needs one question answered honestly.

Reach for Ollama when the model runs on a machine the team controls, breadth of model choice matters more than packaging, the work is exploration or an internal tool, and whoever runs it can be told to install something first.

Reach for Foundry Local when the model has to run on a machine the team does not control, the application is a .NET or cross-platform product shipped to end users, hardware varies across the install base, and the package cannot include an installer for a second product.

A shop can also want both, and that is not indecision. The exploratory loop on a developer's own machine has different constraints from the artifact that reaches a customer. Using a broad tool to find out which model is good enough, then shipping that capability through a library sized for distribution, is a coherent sequence rather than a contradiction.

What makes the choice easy is refusing to treat it as a benchmark question. Tokens per second is the last thing to compare and the first thing everyone reaches for. Start with whose hardware it is. The rest follows.