AI Coding Without API Keys: The Local Stack
AI coding without API keys works when inference, repository context, and agent tools stay on your Mac or Windows PC; the result is a coding loop with no hosted-model credential or per-token meter.
INFERENCE local GGUF model credential: none CONTEXT selected repo files network: local only EDITS Ask → review diff write: after approval CONNECTORS GitHub / Slack OAuth: only when enabled
The credential boundary, line by line
AI coding without API keys removes the hosted model credential from the core reasoning loop. A downloaded model reads selected files, proposes changes, and produces answers locally; only an optional external service needs its own authorization when you deliberately connect it.
The boundary is task-specific. Explaining a function, drafting a unit test, searching a repository, proposing a patch, and reviewing a diff can remain local. Opening a private GitHub issue or reading Slack cannot be anonymous: the service must authorize the request, even though the model doing the reasoning still runs on your machine.
Zimmer AI, built by Fihi Labs UG, runs open-weight models inside a personal workspace on your own Mac or Windows PC; Zimmer Server is the separate multi-user appliance for a company-owned Apple-silicon machine. Inference and working data stay on the machine or private network that owns the workload.
What changed since this guide was published
Zimmer Desktop now ships the same local model and agent harness on macOS and Windows. Windows users can install the x64 or arm64 build without WSL, Docker, Python, or a separate llama.cpp setup; MLX, system-wide dictation, and JJ voice mode remain macOS-only.
The agent surface is also concrete: six built-in roles call nine tools, with up to 15 tool rounds per turn. A delegated subagent gets an isolated context and up to eight rounds. Automatic compaction begins at roughly 80% of the selected context window, while the six most recent turns remain verbatim.
Local model choice has moved quickly. The safe recommendation is no longer a single model name frozen into a guide; it is a current model card plus a memory budget. Qwen now publishes an official Qwen3-Coder-Next GGUF for coding agents, while llama.cpp continues to document broad local backends and low-bit quantization.
Four layers must remain separate
A reliable local coding stack separates inference, context, orchestration, and authority. Combining them into one vague “AI access” switch makes it impossible to tell whether code left the device or whether an agent could write without review.
| Layer | Trusted boundary | Failure to test |
|---|---|---|
| Local model | A GGUF model runs through the bundled llama.cpp server; MLX is available on Apple silicon. | The selected model silently points at a hosted OpenAI-compatible endpoint. |
| Repository context | Only task-relevant files enter the working context. | A whole repository consumes context and exposes unrelated secrets. |
| Agent role | Coder, Reviewer, and Tester roles split implementation from verification. | One long agent turn plans, writes, and approves its own work. |
| Tool permissions | Allow, Ask, and Deny decide whether each action can run. | A tool bypasses the reviewable side-by-side diff. |
Memory budget before model name
Model fit is the first engineering constraint. Zimmer AI treats 16 GB as the practical floor for a 4B-class model at Q4, while 32–36 GB is the comfortable band for a 14B-class model or a larger mixture-of-experts model. Context and development tools need memory too.
| Machine | Start here | Honest workload |
|---|---|---|
| 16 GB Apple-silicon Mac | 4B-class GGUF at Q4, default 16k context | Explanation, focused edits, tests, and short review loops. |
| 32–36 GB Apple-silicon Mac | 14B-class GGUF or larger MoE that fits | Broader repository context and more reliable multi-step work. |
| Below the floor | Do not force a large quantization | A hosted model is cheaper to start and usually more capable. |
The built-in Model Hub classifies downloads as best-for-you, runs-well, possible, or too-large from RAM, chip, and free disk. Q4_K_M is a practical first quantization; Q5_K_M spends more memory for additional precision. Context presets run from 8k to 128k, but the default 16k preserves headroom on smaller machines.
A current model shortlist without fake certainty
Current model recommendations should be filtered by task and file size, not leaderboard position alone. On a 16 GB Apple-silicon Mac, begin with a recent 4B-class instruction or coder model at Q4. At 32–36 GB on a Mac, evaluate a 14B-class model and compare its real diffs against the smaller option.
- Small machine: browse current Qwen or Gemma variants in the Model Hub, then choose the 4B-class Q4 file that lands in best-for-you or runs-well.
- Comfortable machine: test a 14B-class Q4 against your own repository. Keep the smaller model when latency matters more than broader reasoning.
- Large local system: Qwen’s official Qwen3-Coder-Next GGUF model card describes a model built for coding agents and local development. Check the actual quantized file before assuming it fits.
Google’s official CodeGemma documentation remains a useful example of task-specific variants for completion and instruction following. The llama.cpp project documents the quantization and backend layer. Model names decay; memory accounting does not.
macOS and Windows share the local path
Zimmer Desktop uses a normal installer on both supported platforms and bundles the llama.cpp server for GGUF inference. The Mac build runs on Apple silicon. Windows ships as x64 and arm64 NSIS installers; no WSL or Docker is required, and this guide makes no claim about Windows GPU acceleration.
Apple silicon also supports the MLX runtime. System-wide voice dictation and JJ screen-aware voice mode are Mac-only. The coding harness, agent roles, repository context, permission prompts, diff review, automations, and Model Hub are available on Mac and Windows.
Permission gates are part of the architecture
A local model is not automatically a safe agent. Zimmer AI separates reading from mutation with Allow, Ask, and Deny rules. Safe read-only operations can proceed; file edits and commands can require approval; destructive commands such as broad recursive deletion are denied out of the box.
Every proposed file edit appears in a side-by-side Monaco diff before it touches disk. A useful first loop assigns a Reviewer to inspect and plan, hands the approved plan to a Coder, then asks a Tester to derive cases from the final diff. Separate roles create review points instead of letting one agent approve itself.
This is a category-level requirement, not only a Zimmer feature. Claude Code’s permission documentation likewise distinguishes read-only planning from edit authority. Locality controls where inference runs; permissions control what the agent may do.
Offline does not mean every feature is credential-free
Once a GGUF model and the bundled runtime are downloaded, local inference, repository reading, tool calls, document indexing, and agent work can continue without internet access. Downloads, application updates, and any service you deliberately connect still need a network path.
Zimmer’s 33-entry Model Context Protocol catalog uses OAuth 2.1 for services such as GitHub, Slack, Notion, and Linear. That replaces pasted tokens with the vendor’s consent screen; it does not pretend the external service is anonymous. You can leave every connector disabled and keep the coding loop local.
A hosted frontier model from Claude, GPT, or Gemini will outperform any model that fits on a laptop. The local path trades peak capability for privacy, predictable cost, offline continuity, and ownership. If your hardware is below the practical floor, cloud AI is cheaper to start.
Three failure modes to test
A no-key claim is credible only when it survives failure testing. Disconnect the network after setup, inspect the selected endpoint, and force an edit approval. These checks reveal whether “local” describes the model, the application, or only the marketing copy.
- Endpoint drift: verify that the selected model uses the bundled local runtime, not an optional hosted OpenAI-compatible endpoint.
- Context sprawl: select only the files needed for the task and confirm unrelated secrets never enter the prompt.
- Approval bypass: set writes to Ask, request a small edit, and confirm the diff appears before the file changes.
For more detail, compare the agent tool and review model, the hardware-aware AI model hub, and the MCP connector boundary. The offline coding guide covers network-loss planning.
Questions developers ask before switching
Can I use AI for coding without API keys?
Yes. AI coding without API keys works when a downloaded model runs locally and the assistant sends repository context to that local runtime. Code explanation, test drafting, documentation, focused edits, and diff review can avoid hosted model credentials. Optional GitHub, Slack, or other external services still require their own authorization when enabled.
Does AI coding without an API key work offline?
After the model and runtime are downloaded, Zimmer AI can run local inference, read selected project files, call local tools, and generate reviewable edits without internet access. Model downloads, software updates, hosted endpoints, and optional connected services still need a network connection. Offline describes the local path, not every possible integration.
Which local coding model should I use with 16 GB or 32 GB of memory?
On a 16 GB Apple-silicon Mac, start with a 4B-class instruction or coder model at Q4 and keep the default 16k context. With 32–36 GB on a Mac, test a 14B-class Q4 model or a larger mixture-of-experts option. Windows support is live, but this guide makes no claim about Windows GPU acceleration.
How do I stop a local coding agent from changing files without approval?
Set file edits and shell commands to Ask, keep destructive commands denied, and require a visible diff before writes land. In Zimmer AI, the Allow, Ask, and Deny permission tiers control each tool action, while the Monaco diff provides the review point. Use separate Reviewer, Coder, and Tester roles for additional checks.
Run the local path on your machine
Zimmer Desktop is free forever for personal and commercial use. Run open-weight models on one Mac or Windows PC with no hosted model key, no per-token charge, and review before edits land.