AI on your own computer.
What local-first means, and when it is worth it.
What it means to run an AI model on your own machine, what it can and cannot do, and when it is worth the trouble.
§01 · What a local model is
A local AI model is a file of learned numbers that your own computer runs. When you use a hosted chatbot, your words travel to a company's data center and the answer travels back. With a local model, the file is downloaded once, and every later answer is worked out on your own processor, graphics card or neural chip [3][8].
A local model is an AI that runs on your own computer, so the words you give it do not have to leave the device to get an answer [3][8].
You can meet one in three ways. One is a free app you install on purpose, such as Ollama or LM Studio [11][1]. Another is a feature built into the operating system, such as Apple Intelligence on Apple devices or Phi Silica on Windows [9][8]. The third is an ordinary app that may call those system features for small jobs.
Three things follow from running on your own machine. The model has to fit in memory: Ollama loads a model onto the graphics card if it fits, and its ollama ps command shows whether a model runs on the GPU or the CPU [3].
The model is smaller than the hosted ones. Apple describes its built-in model as about 3 billion parameters, compressed to 2 bits per weight, and reports roughly a 4.6 percent drop on one math test from that compression [7].
The model is also frozen: it knows what it learned before its training cutoff [8]. It does not browse the web unless you connect a tool.
§02 · What it can and cannot do
Apple says its on-device model suits summarization, extraction and short dialog, and is not meant to be a general world-knowledge chatbot [7]. Microsoft lists text generation, summarization, rewriting and text-to-table as the skills of its Windows model [8].
| Task | Local model today | What the cloud does better |
|---|---|---|
| Rewrite or tighten your own text | A listed skill of Microsoft's Phi Silica [8] | For short edits, little (our reasoning, not a measured result) |
| Summarize a document you would not upload | Stays on the device in local mode [3][8]; Ollama's default 4096-token window can cut off a long document [3] | Large documents are likely easier on a hosted model (our reasoning, not a measured result) |
| Draft or explain something offline | Apple's features work offline [9] | A local model has a fixed knowledge cutoff [8], so current facts are a weak spot |
| Answer general fact questions | Apple says its model is not meant for this [7] | The strongest hosted models are ahead by the lag described below [5][6] |
| Anything where a mistake is costly | Microsoft says not to use it for high-stakes decisions without human oversight [8] | Hosted models also make mistakes, but they are the stronger option for hard reasoning, given that lag [5][6] |
How far behind are local models? Epoch AI measured that open models small enough for one consumer graphics card match the frontier of about 6 to 12 months earlier [5]. For the largest open models, which most people cannot run at home, the lag is about four months since January 2026 [6]. Epoch warns that real-world lag may be longer than benchmark lag [5].
Mistakes need a plain warning. A local model can give wrong answers that read as sure of themselves. Apple names hallucinations and prompt injection as continuing risks for its own model [7]. Microsoft says its model should not be your sole source of factual truth [8]. Running locally does not make an answer correct.
§03 · What it needs in 2026
There is no single official minimum. Numbers below were read on 2026-11-17 and change quickly.
- Entry, 8 to 16 GB of memory. LM Studio says smaller models may work on an 8 GB Mac, and recommends 16 GB or more [1].
- Mainstream, 16 GB. OpenAI's gpt-oss-20b model is documented to run within 16 GB of memory [2]. For Windows, LM Studio recommends at least 16 GB of RAM and at least 4 GB of dedicated graphics memory [1].
- Enthusiast, 24 to 32 GB of graphics memory. Epoch estimates ceilings of about 27 billion parameters on a 24 GB card and about 40 billion on a 32 GB card. Long outputs leave less headroom [5].
LM Studio's Mac requirement is Apple Silicon (M1 through M4) with macOS 14.0 or newer [1]. Ollama lists support for NVIDIA cards with compute capability 5.0 or higher and driver 550 or newer [10], AMD cards through ROCm v7, Apple devices through Metal, and Vulkan-capable devices [10].
Microsoft's Copilot+ label means a neural chip of over 40 trillion operations per second [12], a threshold for built-in Windows features. Microsoft says Phi Silica also runs on a graphics card in ordinary PCs, with higher latency and more power draw, and no guarantee for integrated graphics or older cards [8]. Memory is the constraint the vendor documents keep coming back to [1][3], so ask how much memory your computer has before anything else.
§04 · Why people do it
Privacy, because in local mode the company running the software does not see your prompts [3][8]. And cost, because running locally has no per-request fee (our reasoning; Apple confirms free inference for developers using its framework [9]).
On privacy, Ollama says it does not see your prompts or data when you run locally [3]. Microsoft says Phi Silica prompts and responses are processed locally and not sent to Microsoft [8]. The claim has limits. Local is a mode, not a brand: Ollama also sells cloud models that run on its servers and need an account [4]. Microsoft cautions against feeding health or financial records to the model without safeguards, even when processing stays on the device [8].
On cost, Apple says inference through its framework is free of cost to developers [9]. What is not free is hardware, electricity and your time. We found no 2026 source that prices a subscription against local hardware, so there is no break-even number here.
Our reasoning: if you already own a machine with 16 GB of memory, the added cost is small. If you would buy a computer mainly for this, the price will usually outweigh light use of a hosted service.
§05 · When it is worth it, and when it is not
Worth it when:
- You want to rewrite or tighten your own text, where you can read the result against the original [8].
- You want to summarize a document you would rather not upload, such as a lease or a bill, and you will check the figures [3][7][8]. Local models are enough for these narrow jobs (our reading of the sources, not a tested result).
- You need drafting or plain-language explanations offline [9][8].
- You already own a machine with enough memory [1][2].
Not worth it when:
- A wrong answer would be expensive: Microsoft advises human oversight for high-stakes decisions [8].
- You need current information, since a local model has a fixed knowledge cutoff [8].
- Your hardware is thin. With 8 GB of memory and no recent graphics card, expect it to be slow, though LM Studio says smaller models may work on an 8 GB Mac [1]. Microsoft warns that integrated graphics or older cards may not meet its minimum [8].
- You would buy hardware just for this.
- You cannot keep it updated: Microsoft recommends installing the latest graphics driver straight from NVIDIA or AMD for non-NPU use [8].
§06 · So what do I do
- 01.
- 02.
Start with what is built in.
Look for Apple Intelligence on a recent Apple device or Phi Silica on Windows [9][8]. Confirm in the settings that the feature stays on the device. Microsoft says Phi Silica is not available in China, and that on a PC without a neural chip it runs on the graphics card (see section 03) [8].
- 03.
Then add one app.
LM Studio gives you a point-and-click window, and Ollama is a free download for macOS, Linux and Windows [11]. Our advice: get installers from lmstudio.ai and ollama.com, never from search ads or mirrors. Model files are multi-gigabyte downloads that can fill a disk, so check your free space (our advice). Running on the graphics card can cut laptop battery life and slow other graphics work [8]. If you want strictly local use, leave Ollama's cloud models off [4].
- 04.
Try a safe job.
Paste in a paragraph you wrote and ask for a tighter version, then compare it with your original. Next, summarize a document and check every figure against the source. For long documents, raise Ollama's context setting above its 4096-token default [3]. Our advice: never paste passwords or credentials, and treat health or financial records with Microsoft's caution [8].
- 05.
- 06.
This publication is written by a small engineering studio; see [the studio](work.html).
Sources
Every claim above points at one of these. They were checked on 2026-11-17. If one stops being true, this page will say so.
- 1.LM Studio: System requirements · Mac, Windows and Linux memory guidancehttps://lmstudio.ai/docs/app/system-requirements
- 2.gpt-oss-20b model card · Runs within 16 GB, Apache 2.0https://huggingface.co/openai/gpt-oss-20b
- 3.Ollama FAQ · Local privacy, 4096 context, GPU loadinghttps://docs.ollama.com/faq
- 4.Ollama Cloud · Cloud models run on its servershttps://docs.ollama.com/cloud
- 5.Epoch AI: consumer hardware model gap · Six to twelve month laghttps://epoch.ai/data-insights/consumer-gpu-model-gap
- 6.Epoch AI: open models lag closed · About four months for largesthttps://epoch.ai/data-insights/open-closed-eci-gap
- 7.Apple: on-device foundation model updates · Size, compression, intended use, riskshttps://machinelearning.apple.com/research/apple-foundation-models-2025-updates
- 8.Microsoft: Phi Silica transparency note · Local processing, limits, GPU fallbackhttps://learn.microsoft.com/en-us/Windows/ai/apis/phi-silica-transparency-note
- 9.Apple Newsroom: Foundation Models framework · Offline, free inference, device requirementshttps://www.apple.com/newsroom/2025/09/apples-foundation-models-framework-unlocks-new-intelligent-app-experiences/
- 10.Ollama GPU documentation · Supported graphics cards and drivershttps://docs.ollama.com/gpu
- 11.Ollama download · Installers for three operating systemshttps://ollama.com/download
- 12.Microsoft Copilot+ PCs · Neural chip threshold, 40 TOPShttps://www.microsoft.com/en-us/windows/copilot-plus-pcs
- A single hardware minimum for Ollama. The pages read describe graphics card support and where models load, not one memory figure.
- Which Apple devices qualify. The pages read say "Apple Intelligence-compatible devices" without a list.
- Whether the 40 TOPS threshold predicts speed in apps like Ollama or LM Studio.
- A cost break-even between a hosted subscription and local hardware and electricity.
- Independent tests of specific popular local models on the jobs above. That local is enough for them rests on vendor-described uses and on the reasoning that rewriting supplied text is an easy task.
- Ollama's cloud retention. Its cloud page gives no retention period, and its FAQ says prompts are not stored or logged.
- Licensing of Ollama and LM Studio for commercial use.
- How many apps use the system models, and which of those stay fully local.
Written from a report in the studio’s research library (RES-4063). Every source was resolved and two independent reviewers tried to refute it before this page was written, then the page was checked again. Last check: 2026-11-17.
How guides get here →