Local AI agents

What a local AI agent is

A local AI agent combines a locally served language model with a tool-calling loop that can take actions on your machine. The agent receives a prompt, decides whether to call a tool or answer directly, observes the result, and repeats until the task is done. The quality of the agent depends on the model's reasoning ability, the tools it can access, and how permissions are configured. A well-configured local agent can edit code, run tests, read documentation, and navigate a repository without a network connection.

How the agent loop works

Most local agents use a ReAct-style loop: the model reads the prompt and conversation history, reasons about what to do next, calls a tool if needed, observes the tool output, and decides whether the task is complete or another step is required. This loop continues until the agent produces a final answer. The key constraint for local agents is context: the model's context window limits how much history, code, and tool output can fit in a single turn, which is why Llanite surfaces context usage in the TUI.

Why run agents locally

Local agents offer several advantages over cloud-based AI agents. All data stays on your machine, which matters for private codebases, sensitive projects, and compliance requirements. There are no per-token API costs, which makes long agentic sessions economical for high-volume development work. Local agents can also work without an internet connection once the model is downloaded, which is useful for offline development, air-gapped environments, or low-bandwidth situations.

Hardware considerations

Local AI agents need enough RAM to load the model and run the agent process alongside other development tools. A 7–9 billion parameter model typically needs 8–12 GB of RAM. A 27 billion parameter coding model needs 20–32 GB. On Apple Silicon Macs, Ollama loads model weights into unified memory shared with the GPU, which improves inference speed but counts against your total RAM budget. Llanite shows a hardware fit rating for each stack so you can see whether your machine is comfortable, tight, or incompatible before installing.

How Llanite packages local agents

Llanite bundles a local agent as a stack: a declared combination of model, runtime, agent layer, tools, permissions, and hardware requirements. Rather than manually configuring an Ollama model, choosing an agent tool, and wiring up permissions, you install a single Llanite stack that has already made those choices. Each stack is inspectable with llanite inspect, so you can see exactly what the agent will be able to do before it runs.

Getting started with a local agent

Setting up a local AI agent with Llanite takes a few steps: install Llanite, confirm Ollama is running, choose a stack that fits your machine, and run it. Llanite's doctor command checks that all prerequisites are in place before you begin.

$ llanite install standard$ llanite run standard

1. 1Install Llanite via npm (Node 20+ required)
2. 2Start Ollama: ollama serve
3. 3Run llanite doctor to confirm Ollama is reachable
4. 4Browse available stacks with llanite stacks
5. 5Install your chosen stack with llanite install <stack>
6. 6Start an agent session with llanite run <stack>