proTECHtusDailyThe publication arm of Protechtus · for every‑day users
Free Guides  /  AI  /  Guide № 08
AIExplainerPrivacy
Explainer · read once

AI on your own computer.
What local-first means, and when it is worth it.

What it means to run an AI model on your own machine, what it can and cannot do, and when it is worth the trouble.

Length
6 min
1505 words
Status
Verified
checked 2026-11-17
Format
Web
Plain words about how it actually works, then what to do.
Sources
12
primary sources, resolved
Price
Free
read and share

§01 · What a local model is

A local AI model is a file of learned numbers that your own computer runs. When you use a hosted chatbot, your words travel to a company's data center and the answer travels back. With a local model, the file is downloaded once, and every later answer is worked out on your own processor, graphics card or neural chip [3][8].

◆ In one sentence

A local model is an AI that runs on your own computer, so the words you give it do not have to leave the device to get an answer [3][8].

You can meet one in three ways. One is a free app you install on purpose, such as Ollama or LM Studio [11][1]. Another is a feature built into the operating system, such as Apple Intelligence on Apple devices or Phi Silica on Windows [9][8]. The third is an ordinary app that may call those system features for small jobs.

Three things follow from running on your own machine. The model has to fit in memory: Ollama loads a model onto the graphics card if it fits, and its ollama ps command shows whether a model runs on the GPU or the CPU [3].

The model is smaller than the hosted ones. Apple describes its built-in model as about 3 billion parameters, compressed to 2 bits per weight, and reports roughly a 4.6 percent drop on one math test from that compression [7].

The model is also frozen: it knows what it learned before its training cutoff [8]. It does not browse the web unless you connect a tool.

§02 · What it can and cannot do

Apple says its on-device model suits summarization, extraction and short dialog, and is not meant to be a general world-knowledge chatbot [7]. Microsoft lists text generation, summarization, rewriting and text-to-table as the skills of its Windows model [8].

TaskLocal model todayWhat the cloud does better
Rewrite or tighten your own textA listed skill of Microsoft's Phi Silica [8]For short edits, little (our reasoning, not a measured result)
Summarize a document you would not uploadStays on the device in local mode [3][8]; Ollama's default 4096-token window can cut off a long document [3]Large documents are likely easier on a hosted model (our reasoning, not a measured result)
Draft or explain something offlineApple's features work offline [9]A local model has a fixed knowledge cutoff [8], so current facts are a weak spot
Answer general fact questionsApple says its model is not meant for this [7]The strongest hosted models are ahead by the lag described below [5][6]
Anything where a mistake is costlyMicrosoft says not to use it for high-stakes decisions without human oversight [8]Hosted models also make mistakes, but they are the stronger option for hard reasoning, given that lag [5][6]

How far behind are local models? Epoch AI measured that open models small enough for one consumer graphics card match the frontier of about 6 to 12 months earlier [5]. For the largest open models, which most people cannot run at home, the lag is about four months since January 2026 [6]. Epoch warns that real-world lag may be longer than benchmark lag [5].

Mistakes need a plain warning. A local model can give wrong answers that read as sure of themselves. Apple names hallucinations and prompt injection as continuing risks for its own model [7]. Microsoft says its model should not be your sole source of factual truth [8]. Running locally does not make an answer correct.

§03 · What it needs in 2026

There is no single official minimum. Numbers below were read on 2026-11-17 and change quickly.

  • Entry, 8 to 16 GB of memory. LM Studio says smaller models may work on an 8 GB Mac, and recommends 16 GB or more [1].
  • Mainstream, 16 GB. OpenAI's gpt-oss-20b model is documented to run within 16 GB of memory [2]. For Windows, LM Studio recommends at least 16 GB of RAM and at least 4 GB of dedicated graphics memory [1].
  • Enthusiast, 24 to 32 GB of graphics memory. Epoch estimates ceilings of about 27 billion parameters on a 24 GB card and about 40 billion on a 32 GB card. Long outputs leave less headroom [5].

LM Studio's Mac requirement is Apple Silicon (M1 through M4) with macOS 14.0 or newer [1]. Ollama lists support for NVIDIA cards with compute capability 5.0 or higher and driver 550 or newer [10], AMD cards through ROCm v7, Apple devices through Metal, and Vulkan-capable devices [10].

Microsoft's Copilot+ label means a neural chip of over 40 trillion operations per second [12], a threshold for built-in Windows features. Microsoft says Phi Silica also runs on a graphics card in ordinary PCs, with higher latency and more power draw, and no guarantee for integrated graphics or older cards [8]. Memory is the constraint the vendor documents keep coming back to [1][3], so ask how much memory your computer has before anything else.

§04 · Why people do it

◆ Two honest reasons

Privacy, because in local mode the company running the software does not see your prompts [3][8]. And cost, because running locally has no per-request fee (our reasoning; Apple confirms free inference for developers using its framework [9]).

On privacy, Ollama says it does not see your prompts or data when you run locally [3]. Microsoft says Phi Silica prompts and responses are processed locally and not sent to Microsoft [8]. The claim has limits. Local is a mode, not a brand: Ollama also sells cloud models that run on its servers and need an account [4]. Microsoft cautions against feeding health or financial records to the model without safeguards, even when processing stays on the device [8].

On cost, Apple says inference through its framework is free of cost to developers [9]. What is not free is hardware, electricity and your time. We found no 2026 source that prices a subscription against local hardware, so there is no break-even number here.

Our reasoning: if you already own a machine with 16 GB of memory, the added cost is small. If you would buy a computer mainly for this, the price will usually outweigh light use of a hosted service.

§05 · When it is worth it, and when it is not

Worth it when:

  • You want to rewrite or tighten your own text, where you can read the result against the original [8].
  • You want to summarize a document you would rather not upload, such as a lease or a bill, and you will check the figures [3][7][8]. Local models are enough for these narrow jobs (our reading of the sources, not a tested result).
  • You need drafting or plain-language explanations offline [9][8].
  • You already own a machine with enough memory [1][2].

Not worth it when:

  • A wrong answer would be expensive: Microsoft advises human oversight for high-stakes decisions [8].
  • You need current information, since a local model has a fixed knowledge cutoff [8].
  • Your hardware is thin. With 8 GB of memory and no recent graphics card, expect it to be slow, though LM Studio says smaller models may work on an 8 GB Mac [1]. Microsoft warns that integrated graphics or older cards may not meet its minimum [8].
  • You would buy hardware just for this.
  • You cannot keep it updated: Microsoft recommends installing the latest graphics driver straight from NVIDIA or AMD for non-NPU use [8].

§06 · So what do I do

  1. 01.

    Check your memory.

    Find out how much memory your computer has. Sixteen gigabytes is a realistic entry point as of 2026-11-17 [1][2].

  2. 02.

    Start with what is built in.

    Look for Apple Intelligence on a recent Apple device or Phi Silica on Windows [9][8]. Confirm in the settings that the feature stays on the device. Microsoft says Phi Silica is not available in China, and that on a PC without a neural chip it runs on the graphics card (see section 03) [8].

  3. 03.

    Then add one app.

    LM Studio gives you a point-and-click window, and Ollama is a free download for macOS, Linux and Windows [11]. Our advice: get installers from lmstudio.ai and ollama.com, never from search ads or mirrors. Model files are multi-gigabyte downloads that can fill a disk, so check your free space (our advice). Running on the graphics card can cut laptop battery life and slow other graphics work [8]. If you want strictly local use, leave Ollama's cloud models off [4].

  4. 04.

    Try a safe job.

    Paste in a paragraph you wrote and ask for a tighter version, then compare it with your original. Next, summarize a document and check every figure against the source. For long documents, raise Ollama's context setting above its 4096-token default [3]. Our advice: never paste passwords or credentials, and treat health or financial records with Microsoft's caution [8].

  5. 05.

    Know what not to trust it with.

    Do not use it as the sole source for medical, legal or financial decisions [8]. Do not ask it for current events or facts you cannot check, because it can be wrong [7][8].

  6. 06.

    Keep a hosted model for hard reasoning.

    Local models trail the strongest hosted ones, by months, and sometimes by more [5][6].

This publication is written by a small engineering studio; see [the studio](work.html).

Sources

Every claim above points at one of these. They were checked on 2026-11-17. If one stops being true, this page will say so.

  1. 1.LM Studio: System requirements · Mac, Windows and Linux memory guidancehttps://lmstudio.ai/docs/app/system-requirements
  2. 2.gpt-oss-20b model card · Runs within 16 GB, Apache 2.0https://huggingface.co/openai/gpt-oss-20b
  3. 3.Ollama FAQ · Local privacy, 4096 context, GPU loadinghttps://docs.ollama.com/faq
  4. 4.Ollama Cloud · Cloud models run on its servershttps://docs.ollama.com/cloud
  5. 5.Epoch AI: consumer hardware model gap · Six to twelve month laghttps://epoch.ai/data-insights/consumer-gpu-model-gap
  6. 6.Epoch AI: open models lag closed · About four months for largesthttps://epoch.ai/data-insights/open-closed-eci-gap
  7. 7.Apple: on-device foundation model updates · Size, compression, intended use, riskshttps://machinelearning.apple.com/research/apple-foundation-models-2025-updates
  8. 8.Microsoft: Phi Silica transparency note · Local processing, limits, GPU fallbackhttps://learn.microsoft.com/en-us/Windows/ai/apis/phi-silica-transparency-note
  9. 9.Apple Newsroom: Foundation Models framework · Offline, free inference, device requirementshttps://www.apple.com/newsroom/2025/09/apples-foundation-models-framework-unlocks-new-intelligent-app-experiences/
  10. 10.Ollama GPU documentation · Supported graphics cards and drivershttps://docs.ollama.com/gpu
  11. 11.Ollama download · Installers for three operating systemshttps://ollama.com/download
  12. 12.Microsoft Copilot+ PCs · Neural chip threshold, 40 TOPShttps://www.microsoft.com/en-us/windows/copilot-plus-pcs
What we could not confirm
  • A single hardware minimum for Ollama. The pages read describe graphics card support and where models load, not one memory figure.
  • Which Apple devices qualify. The pages read say "Apple Intelligence-compatible devices" without a list.
  • Whether the 40 TOPS threshold predicts speed in apps like Ollama or LM Studio.
  • A cost break-even between a hosted subscription and local hardware and electricity.
  • Independent tests of specific popular local models on the jobs above. That local is enough for them rests on vendor-described uses and on the reasoning that rewriting supplied text is an easy task.
  • Ollama's cloud retention. Its cloud page gives no retention period, and its FAQ says prompts are not stored or logged.
  • Licensing of Ollama and LM Studio for commercial use.
  • How many apps use the system models, and which of those stay fully local.
Where this comes from

Written from a report in the studio’s research library (RES-4063). Every source was resolved and two independent reviewers tried to refute it before this page was written, then the page was checked again. Last check: 2026-11-17.

How guides get here →
Next on the shelf
Setting up a phone for someone you love.
Guide № 09 · Web + print · 6 min →
✉ The Dispatch

An email when there is something worth sending.
Honest, brief, forwardable.

Scam shapes, one explainer, and one thing worth printing out, when there is an issue worth sending. Sign-ups are open; the first issue goes out when there is one.

Free · no fixed schedule · unsubscribe anytime