NVIDIA GeForce RTX 3090

If you want 24GB of VRAM for running LLMs at home and you don’t want to spend flagship money, the used RTX 3090 is the card I keep coming back to. It’s the ex-mining GPU reborn as the local-AI value king. For anyone building their first serious inference box on a budget, this is where I’d start.

The take

Buy it for the VRAM, not the speed. Twenty-four gigabytes of GDDR6X at 936 GB/s is what matters for local models, and the 3090 hands you that for roughly the price of a new mid-range card. It’s not the fastest 24GB option going, and it runs hot and hungry at 350W. But pound for pound of usable VRAM, nothing current touches it. I run 3090-class cards on my bench precisely because of this maths.

What it’ll actually run

Twenty-four gigabytes is the sweet spot for single-card local AI. You’ll comfortably run a 32B model at Q4 with a decent context window, and that’s the class of model most people actually want day to day. Drop to a 4-bit 13B or a Q4 quant of something like Qwen or Llama in the 8B to 14B range and it flies, sitting fully in VRAM.

Expect real numbers in the region of 30 to 45 tokens/sec on a 13B at Q4, and something like 15 to 25 tokens/sec on a 32B, depending on quant and context. That’s comfortably faster than you can read. Push to a 70B and you’re into low-quant territory with CPU offload, which drags speed down hard; if 70B is the goal, get a second 3090 and split it.

The value lens is £/GB-VRAM. At around £650 for 24GB you’re paying roughly £27 per gigabyte. Very little else on the used or new market gets close, and that’s the whole argument for this card. Software-wise you’re on mature CUDA drivers that every inference stack already targets, so setup with Ollama, llama.cpp or vLLM is a non-event.

Who should buy it, and who shouldn’t

Buy it if you want maximum VRAM for the money and you’re comfortable with the used market. That means checking seller history, asking for proof it works, and budgeting an afternoon to test memory and re-pad if it ran hot in a mining rig. Tinkerers and first-time local-AI builders get the most out of it.

Skip it if you need a warranty, a quiet small-form-factor machine, or low idle power for an always-on assistant; the 350W ceiling and triple-slot coolers aren’t friendly there. If new-with-support matters more than value, look at a 5090 for raw speed and 32GB, or a used 4090 if you want 24GB with more grunt. But for the person asking “what’s the cheapest way into serious local LLMs”, the answer is still a carefully chosen 3090.