AI · GPU comparison

The best GPUs for local AI, ranked

For running models at home, VRAM decides what fits and bandwidth decides how fast it answers - gaming benchmarks barely come into it. Here's the whole field, ranked by our own weighted score, with price per gigabyte alongside as the value check.

How we rank. Top down: VRAM first, because it decides what you can load at all, then memory bandwidth, because that sets how fast the answers come. Our own weighted score - VRAM value, tokens per second, efficiency, drivers, thermals, longevity - sits in its own column, so sort by that, or by price per gig, whenever you'd rather.

Sort:

#	GPU	Score	VRAM	Bandwidth	~Q4 fit	£/GB
1	NVIDIA RTX PRO 6000 Blackwell workstation	3.9	96GB	1792 GB/s	~188B	£78
2	NVIDIA RTX A6000 workstation · used	3.6	48GB	768 GB/s	~92B	£67
3	NVIDIA GeForce RTX 5090 flagship	3.9	32GB	1792 GB/s	~60B	£59
4	NVIDIA GeForce RTX 3090 Ti value · used	3.5	24GB	1008 GB/s	~44B	£29
5	NVIDIA GeForce RTX 4090 flagship · used	4	24GB	1008 GB/s	~44B	£58
6	AMD Radeon RX 7900 XTX upper-mid	3.1	24GB	960 GB/s	~44B	£35
7	NVIDIA GeForce RTX 3090 value · used	3.7	24GB	936 GB/s	~44B	£27
8	NVIDIA Tesla P40 workstation · used	2.7	24GB	347 GB/s	~44B	£10
9	NVIDIA GeForce RTX 5080 upper-mid	3.7	16GB	960 GB/s	~28B	£63
10	NVIDIA GeForce RTX 5070 Ti upper-mid	3.8	16GB	896 GB/s	~28B	£44
11	NVIDIA GeForce RTX 4070 Ti SUPER value	3.7	16GB	672 GB/s	~28B	£38
12	NVIDIA GeForce RTX 4060 Ti 16GB budget	3.9	16GB	288 GB/s	~28B	£25
13	Intel Arc B580 budget	3.4	12GB	456 GB/s	~20B	£21
14	NVIDIA GeForce RTX 3060 12GB budget · used	3.7	12GB	360 GB/s	~20B	£19
15	NVIDIA GeForce RTX 3080 value · used	3.3	10GB	760 GB/s	~16B	£35

Score = our weighted rubric out of 5. ~Q4 fit = largest dense model, roughly, at 4-bit. Prices indicative; cards link through to the full review.

The datacentre ceiling

The workstation cards - the A6000 and the 96GB PRO 6000 - now sit in the table above, ranked with the rest. Above even those are the datacentre parts. Nobody's buying an H200 for the study, but they're the reference ceiling: this is what the frontier runs on.

Card	VRAM	Memory	Bandwidth	Notes
NVIDIA H100 reference	80GB	HBM3	3350 GB/s	Datacentre part. Enormous bandwidth, rarely a sane home buy.
NVIDIA H200 reference	141GB	HBM3e	4800 GB/s	Frontier inference. Reference only - the top of the mountain.

What will it actually run?

Try the VRAM calculator →

Model size	VRAM at Q4	Cheapest card that clears it
3-4B	~3-4 GB	Any 8GB+ card
7-8B	~5-6 GB	RTX 3060 12GB
13-14B	~9-10 GB	RTX 3060 12GB / 4060 Ti 16GB
27-32B	~18-20 GB	RTX 3090 / 4090 / 5090 (24GB+)
70B	~40 GB	Dual 24GB, RTX PRO 6000, or a 128GB AI PC
120B+ MoE	~60-70 GB	RTX PRO 6000 96GB / multi-GPU / big unified memory

Rule of thumb: a model needs roughly half a gigabyte of VRAM per billion parameters at 4-bit, plus a little headroom for context. Go over your VRAM and the overflow spills into system RAM, which works but slows right down. If a used RTX 3090 fits your models, it's still the value pick of the whole table.

Prefer a prebuilt box? → AI PCs Best prebuilt AI PC →