Live now · Get your API key

Local models and routing

How Ambient Desktop routes work across local llama.cpp runtimes, network models, and cloud providers, keeping private, cheap, or latency-sensitive work on your machine.

Ambient Desktop chooses between local and remote models for each step of a task, keeping private, cheap, or latency-sensitive work on your machine.

Ambient Desktop settings and workspace.
Search, scraping, browser, model, and media providers can be prioritized and retried according to user preference and task fit.

One task, the right model#

A single model is rarely the right answer for every step of a task. Summarizing a private file, analyzing a screenshot, and reasoning through an architecture trade-off have different needs for privacy, latency, cost, and capability. Ambient Desktop treats the model layer like its provider layer: an ordered set of runtimes that can be selected and retried by what the task actually needs. Not every task should leave your machine.

TASK STEPROUTERCHOSEN BY TASK FITOne stepsummarize · look · reasonRouterordered · retryableprioritize local-firstdisable · inspect evidenceLocal runtimellama.cpp · private, cheap, offlineLocal visionMiniCPM-V · pixels stay on deviceHeavy reasoningcloud / network models (Ambient)Ambient Miniplanned local-model network layer

A routing diagram. On the left, one step of a task (summarize, look, or reason) flows into a central Router node in the middle. The router is ordered and retryable, and you can prioritize local-first, disable a provider, or inspect its evidence. From the router, four arrows fan out to the runtime classes it can choose by task fit: a local llama.cpp runtime for private, cheap, or offline work; a local MiniCPM-V vision runtime where pixels stay on device; heavy reasoning on cloud or network models such as Ambient; and the planned Ambient Mini local-model network layer.

Ambient Desktop treats the model layer like its provider layer: each task step enters an ordered, retryable router that picks the runtime the step actually needs: local, local vision, heavy cloud or network reasoning, or the planned Ambient Mini layer.

What can route where#

  • Local runtimes: llama.cpp-hosted models with non-destructive start/stop lifecycle management, for private, low-cost, or offline-friendly work.
  • Vision, locally: a local MiniCPM-V runtime so screenshots and images can be understood even when the selected text model has no native vision. See Local visual analysis.
  • Heavy reasoning: high-capability cloud and network-native models (Ambient or another selected provider) for the hard synthesis steps a small local model should not own.
  • Ambient Mini (planned): a local-model network layer where eligible machines serve and earn, built on this same runtime foundation. See Mini Miner.

Controls you have#

  • Prioritize local-first, fastest, most private, cheapest, or most capable providers.
  • Disable a provider entirely so it never receives a class of content.
  • Inspect fallback and skip evidence to see why a given runtime was or was not used.
  • Repair or revalidate local runtimes from Settings when assets or binaries drift.

Frequently asked#

How is this different from provider routing? Provider routing covers search, fetch, browser, and vision capabilities; local model routing is the model layer specifically. They share the same idea: an ordered, retryable set chosen by preference and task fit.

Do I need a powerful machine? Local runtimes scale with your hardware. You can keep heavy reasoning in the cloud and use local models for private, cheap, or vision tasks, or run more locally on capable machines.

Keep going#