GPU for local LLM · upper-mid
AMD Radeon RX 7900 XTX
24GB for less than the green team, if you'll put up with ROCm instead of CUDA.
- VRAM
- 24 GB
- TDP
- 355 W
- Bandwidth
- 960 GB/s
- £ / GB
- £35
What it runs
With 24 GB you can comfortably load, at 4-bit quantisation, a dense model up to roughly
~44B params
Verdict
24GB of new-with-warranty VRAM at a fair price, as long as you accept ROCm's rough edges.
BEST FOR
- + 24GB on a new card
- + llama.cpp and LM Studio users
- + people who like tinkering
NOT FOR
- - plug-and-play CUDA workflows
- - training and niche AI tools
- - anyone who wants zero friction
The Radeon RX 7900 XTX is the 24GB card for people who don’t want to pay NVIDIA prices and don’t mind rolling their sleeves up. New, with a warranty, it gives you the same model-size headroom as a 3090 for a fair outlay. The price of admission is ROCm instead of CUDA, and that’s the whole story of this card. Get past it and it’s a lot of VRAM for the money.
The take
For running LLMs, the 7900 XTX works. That surprises people who’ve only heard the CUDA gospel. LM Studio and llama.cpp support it, a 24GB buffer runs proper models, and the memory bandwidth is right up there. The catch is the software stack. Almost everything in local AI assumes CUDA first, so ROCm means occasional setup pain, Windows support that trails Linux, and tools that simply expect an NVIDIA card. If tinkering is part of the fun, this is a bargain. If you want zero friction, it isn’t your card.
What it’ll actually run
Twenty-four gigabytes puts it in the same bracket as a 3090. A 32B at Q4 sits in VRAM with a usable context, and 7B to 14B models run comfortably with room to spare. For everyday assistant and coding work the capacity is there and the card holds its own.
Speed is where paper and practice diverge. On specs the 7900 XTX has strong bandwidth and high FP16 throughput, but real token rates lean on the software path you get working. A cleanly supported CUDA card often matches it despite lower numbers. So set expectations at roughly 3090-class for inference, and treat any backend that runs faster as a bonus. Two cards pool 48GB for a 70B, though multi-GPU on ROCm asks for more patience than the NVIDIA route.
On the hardware itself there’s little to fault. The triple-fan reference cooler keeps a 355W card in check, it takes two ordinary 8-pin connectors, and it drops into a normal build. No 16-pin worries here.
Who should buy it, and who shouldn’t
Buy it if you want 24GB on a new card at a sensible price and you’re comfortable with ROCm, or happy to fall back to the Vulkan runtime when ROCm sulks. It suits Linux users and tinkerers who mostly want inference through LM Studio or llama.cpp and like the idea of not paying the NVIDIA tax.
Skip it if your workflow leans on CUDA-only tools, if you plan to train rather than just infer, or if you want it to work first time with no reading. Those are all reasons a used 3090 or a new 5070 Ti earns its premium. But for inference on a budget with 24GB and a warranty, the 7900 XTX is a fair deal with an asterisk.
Settings people actually run
The configs owners land on, pulled from the community. A sensible starting point, not gospel - tune to your own kit.
Single-card 32B
A 32B at Q4_K_M in LM Studio via ROCm or Vulkan
Fits inside 24GB with context; performance depends on which backend you get working cleanly.
Windows setup
LM Studio with the ROCm runtime, or the Vulkan runtime as a fallback
Vulkan is the easy path when ROCm on Windows fights you; ROCm on Linux is the more complete route.
70B across two cards
Two 7900 XTXs, 48GB pooled, Q4_K_M
Owners have run a 70B split across two cards, though multi-GPU on ROCm needs more care than the CUDA equivalent.
What owners report
Real first-hand experience gathered from owners and the community.
- “
Owners running dual 7900 XTX cards have driven a 70B model across the pair, noting PCIe bandwidth and CPU can bottleneck tokens per second on some platforms.
2 claim(s) we couldn't fully verify
- · indicative price around GBP 850 / USD 950 (mid-2026) - No primary source (AMD or TechPowerUp) tracks street pricing; figure is indicative only.
- · real-world inference roughly 3090-class despite higher paper bandwidth - Depends on ROCm/Vulkan backend and model; community benchmarks vary widely.
Hands-on reviews we drew on
We don't just copy the spec sheet. These are the teardowns and hands-on reviews behind this page - worth watching in their own right.
▶Country Boy Computers
7900 XTX + Instinct MI50-32gb: AMD's Unholy LLM Alliance
▶Country Boy Computers
Dual Radeon RX 7900 XTX running Deepseek-R1:70b
- AMD Radeon RX 7900 XTX SpecsTechPowerUp · primary source
Common questions
Can you actually run local LLMs on a 7900 XTX?+
Yes, and for plain inference it works well. LM Studio and llama.cpp support the card through ROCm and Vulkan, and a 24GB 7900 XTX runs the same model sizes as a 3090. The friction is setup and the wider software stack, not whether text generation works day to day.
How bad is the ROCm situation compared to CUDA?+
It's the real trade-off. CUDA is the default every project targets, so NVIDIA cards just work. ROCm has improved a lot and covers mainstream inference, but you'll hit tools that assume CUDA, Windows support that lags Linux, and the odd setup that needs specific driver versions. If you enjoy solving that, fine. If you want it to just work, buy NVIDIA.
Is it faster than a 3090?+
On paper the 7900 XTX has higher memory bandwidth, 960 GB/s against 936, and strong FP16 throughput. In practice real-world token speed depends heavily on the software path, and a well-supported CUDA card often matches or beats it despite the numbers. Treat it as roughly 3090-class for inference, not clearly ahead.