Skip to Content

Local LLM without a graphics card: what holds, what crashes

A 30-billion-parameter model on a mini-PC with integrated graphics: measurements, abandoned options and crashes.
November 16, 2026 by
Local LLM without a graphics card: what holds, what crashes
Federico Solutions, Myrko Federico

Local LLM without a graphics card: what holds, what crashes

Field: local AI, language models, hardware

Period: August to September 2026

Client: internal project, my own companies (Lab 77, R&D)

Running a 30-billion-parameter language model on a mini-PC with no dedicated graphics card, using only its integrated graphics processor: I did it from August to September 2026, for voice commands and a document assistant. Part of it holds; another part crashes, for a reason still unknown. Here is what I measured, what I abandoned, and what I keep local.

Context

In my article on local LLMs, I explain that memory decides everything, and that my assistants' general-purpose reasoning goes through an online model. This article describes the reverse experiment: handing whatever can be handed over to a local model. An integrated graphics processor has no memory of its own: the model lives in the machine's RAM, shared with the processor. To serve the model, I use NoLlama, an open source server (MIT licence) designed for this kind of hardware, built on OpenVINO GenAI (Apache 2.0 licence). The same model later powered the assistant of my offline library.

Problem

  • Answers too slow: at first, a voice command took 18 to 29 seconds to complete, and one in four failed.
  • A fragile format: an assistant that controls devices must produce exact tool calls. At the slightest format deviation, the command becomes text read out loud.
  • Shared memory: everything running on the machine competes for the same resources.
  • Crashes: the graphics processor locked up several times, without warning.

The approach

1. Measure before choosing the model

My intuition: add a small, faster model for voice. Measurement proved it wrong. On the same protocol, a dense 8-billion-parameter model answered in 19.0 seconds on the processor and 9.3 seconds on the graphics chip; Qwen3-30B-A3B, a 30-billion "mixture of experts" model that activates only about 3 billion for each word, answered in 4.8 seconds. The big one was the fastest; a second benchmark confirmed it: 2.7 seconds, against 5.3 for a dense 4-billion model.

2. Fix the real bottlenecks

Three adjustments did most of the work. First, the "Instruct" variant of the model (Apache 2.0 licence), which does not reason out loud before answering. Next, a fix to the server, which did not recognise a slightly different tool-call format produced by this model: my change accepts that format only for a tool actually declared. Finally, an enlarged memory reserve for the beginning of requests already processed: when it is used, the time to the first word drops from about fifteen seconds to around 200 milliseconds.

3. Abandon what did not hold

Two large models loaded at the same time: failure, tested twice, while plenty of memory remained available. The 30-billion model on the processor alone: too slow. The chip dedicated to AI: ruled out, because the server does not allow tool calls on it and caps requests at 4,096 tokens, while those of the voice assistant exceed 20,000. A more recent model that reasons by default: 38 seconds to say hello, and no tool call.

4. Look for the cause of the crashes without jumping to conclusions

During a crash, every generation fails, while the server's health check still reports "ready". The system log counts 8 episodes over 35 days, 6 of them in a single week. My first explanation, requests that were too long, then the second, too high a volume, were refuted by measurement: one day, the machine served 1,831 generations, with requests of up to about 2,100 tokens, without a single incident. A crash also occurred on an almost idle machine, which rules out processor load as the sole cause. The cause is not established. The lead being followed is the graphics driver: a first update was applied in mid-September, and I have nothing that lets me claim it solves the problem.

Result

A voice command completes in 2.6 to 4.1 seconds, against 18 to 29 at the start, and four commands out of four were executed in the validation test, against three before. A single model serves everything. Building the search indexes now runs in a separate process, on the processor: restarting the model after a crash no longer interrupts indexing.

What I take away

  • A mixture-of-experts model changes the game on modest hardware. The speed of a small model, the knowledge of a large one, if there is enough memory to hold it.
  • A health check must generate. A service that answers "ready" without being able to produce a sentence is misleading. I trust only a real generation.
  • Keep local what is short and well defined. Voice commands, brief answers drawn from documents, sorting book records: the local model copes. On long conversation requests, I measured about 15 seconds before the answer; handing them to an online model remains a trade-off between speed and offline autonomy.
  • Write "cause not established" when that is the case. A hypothesis recorded as a fact ends up guiding the next decisions.

Going further

Would you like to know what a local model can do on the hardware you already have, before buying a graphics card? I measure on your real cases, then size the system. See sovereign AI, or book a 20-minute discovery call.

Sources

Tags: local AI, LLM, integrated graphics, mixture of experts, NoLlama, OpenVINO, Qwen3, measurement, sovereign AI

Measure before you sell: the test bench for my AI models
An exam set before training, a machine that does not pick its winner, written verdicts.