AI PC · workstation
NVIDIA DGX Spark
A 128GB desk-side box for running models a 24GB card can't hold - just don't expect speed.
- Memory
- 128 GB
- Bandwidth
- 273 GB/s
- AI compute
- 1000 TOPS
- £ / GB
- £27
What it runs
With 128 GB of memory you can load, at 4-bit, a model up to roughly
~252B params
Verdict
Buy it for the 128GB of memory and the native CUDA stack, not for tokens per second.
BEST FOR
- + running very large models locally
- + developers who need CUDA
NOT FOR
- - fast everyday chat
- - value per pound
The DGX Spark is Nvidia’s answer to a question most people didn’t know they could ask: what if a little box on your desk held 128GB of memory that a model could live inside? It’s aimed squarely at developers and researchers who want to load large models at home, on Nvidia’s own software stack, without renting cloud time by the hour.
The take
Buy it for the memory, not the speed. That’s the whole story in one line. The 128GB of unified memory lets you load models a 24GB consumer card simply can’t hold, and you get the full Nvidia CUDA stack to do it on. But the memory runs at around 273 GB/s, and a used RTX 3090 moves data at 936 GB/s. So on any model small enough to fit in 24GB, the humble 3090 will answer faster. The Spark is a machine for running big, not for running quick, and it’s worth being clear-eyed about that before the money leaves your account.
What it’ll actually run
This is where the 128GB earns its keep. A 70B model at Q4 needs roughly 40GB, so it drops in with masses of room for a long context window. You can go bigger still - 120B-class models fit comfortably, and you can run a 70B at a higher precision than any single consumer GPU would allow. If the goal is loading a model that a 3090 flat-out can’t, the Spark delivers every time.
The catch is speed, and it comes back to bandwidth. All that memory is fed at around 273 GB/s, which is modest, so tokens land at a steady walking pace rather than a sprint. On a 70B at Q4 you’re realistically looking at single figures to low double figures of tokens per second, depending on context. Quick enough to work with, not quick enough to feel instant. The headline 1,000 AI TOPS of FP4 compute is enormous, but on local inference it’s the bandwidth that sets the ceiling, not the maths. The software side is the real treat: DGX OS is Ubuntu underneath, CUDA works natively, and every framework you’d reach for is already at home here.
Who should buy it
Buy it if you’re a developer or researcher who needs to prototype large models locally, wants the native Nvidia toolchain, and would rather own the hardware than watch a cloud meter tick. For that person it’s a lovely, quiet, self-contained lab.
Give it a miss if what you actually want is a fast everyday assistant. If your models fit inside 24GB, a used 3090 runs them quicker and costs a fifth as much - and two 3090s will handle a 70B at Q4 with more pace than the Spark for less outlay, if you don’t mind the build. The Spark’s argument is capacity and software, not value or raw speed. Match it to the job and you’ll love it. Buy it expecting a speed demon and you’ll be disappointed.
Settings people actually run
The configs owners land on, pulled from the community. A sensible starting point, not gospel - tune to your own kit.
Biggest model that fits
gpt-oss-120b MXFP4 (~59GB)
Loads with room for a long context; MoE design keeps it near 50-58 tokens/s single-stream despite the size.
Fast single-user chat
gpt-oss-20b MXFP4
Around 80 tokens/s on llama.cpp; the everyday sweet spot.
Agentic / concurrent throughput
vLLM with NVFP4, batched requests
Single-stream speed is modest, but aggregate throughput scales past 1,000 tokens/s at high concurrency.
What owners report
Real first-hand experience gathered from owners and the community.
- “
llama.cpp maintainer benchmarks: gpt-oss-120b MXFP4 runs about 58 tokens/s single-stream, falling to roughly 43 at 32k context; gpt-oss-20b about 83; but a dense Qwen2.5-7B only manages about 29. Mixture-of-experts models fly, dense models drag.
- “
Single-user tokens/s on a 30B-class model lands around 50-76 on llama.cpp; the machine's real advantage is concurrency. Measured roughly 1,125 tokens/s aggregate under vLLM at 64 concurrent requests, and about 1,573 with NVFP4.
- “
The 273 GB/s memory bandwidth, not the 1,000 TOPS of compute, sets the single-stream ceiling. Quiet in use; ships with a 240W external PSU while the GB10 SoC itself is rated at 140W.
1 claim(s) we couldn't fully verify
- · A dense 70B at Q4 sits in single figures of tokens per second - No public dense-70B single-stream benchmark on the Spark was found; inferred from the 273 GB/s bandwidth and the measured dense-vs-MoE gap, so it is an estimate rather than a measured figure.
Hands-on reviews we drew on
We don't just copy the spec sheet. These are the teardowns and hands-on reviews behind this page - worth watching in their own right.
▶Alex Ziskind
I Thought DGX Spark Was Slower... Until I Changed ONE Thing
▶LMSYS Org
NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
▶Bijan Bowen
NVIDIA DGX Spark - A Non-Sponsored Review (Strix Halo Comparison, Pros and Cons)
- Performance of llama.cpp on NVIDIA DGX Sparkggerganov (llama.cpp) · primary source
- DGX Spark Hardware Overview (vendor spec)NVIDIA · primary source
- Optimizing GPT-OSS on NVIDIA DGX SparkLMSYS Org · review
Common questions
How does the DGX Spark compare to an RTX 3090 for local LLMs?+
Different jobs. The Spark has 128GB of memory so it loads models a 24GB 3090 can't touch. But its memory runs at around 273 GB/s versus the 3090's 936 GB/s, so on any model that fits in 24GB the 3090 answers faster. Big models go to the Spark, speed goes to the 3090.
What size model can the DGX Spark run?+
A 70B at Q4 needs roughly 40GB, so it fits with plenty of room for context. You can go further into 120B-class models, or run a 70B at higher precision than any single consumer GPU allows. Capacity is the whole point of the machine.
Why is the DGX Spark slower than its compute figure suggests?+
Local inference is limited by memory bandwidth, not raw maths. The 1,000 AI TOPS of FP4 compute is enormous, but at around 273 GB/s the memory feeds the chip slowly, so tokens land at a walking pace on large models.