How to undervolt a GPU - and why I power-test every card I run

|   0  NVIDIA GeForce RTX 4090      WDDM  |   00000000:01:00.0  On |                  Off |
| 39%   47C    P2            108W /  370W |   47774MiB /  49140MiB |    100%      Default |

The interesting pair is 100% utilisation against 108W of a 370W cap. If that card were gaming, those two numbers would move together - full utilisation would mean something close to full power. On inference they've come apart completely, and the reason matters for every decision downstream: running a language model is limited by memory bandwidth, not compute. The GPU cores spend most of their time waiting for weights to arrive from VRAM. Waiting is cheap. So the card reports itself busy while sipping.

The practical consequence: whatever power limit your card ships with, a 24/7 inference rig will rarely need it, and the headroom above what the workload uses buys you nothing except heat and noise and bill. That's the free money. The rest of this article is about collecting it without hurting anything.

Why watts became the number I watch

I learned to care about this the expensive way, back when I ran a rack of cards flat out around the clock and the electricity bill was the whole argument. I could tell you what my PC cost per hour to run (8p, at the time), and every card had been through the same ritual: drop the power limit, re-run the same job, find the point where the work stops falling as fast as the watts do. Curves in Afterburner were half the hobby.

I assumed that discipline was retired along with the rack. It wasn't. Local AI has much the same load profile - memory-bound, always-on, thermally relentless - and the testing habits transfer almost untouched. If anything the case is stronger now, because an inference card is often on around the clock whether or not anyone is using it.

Power cap vs undervolt - two doors, same room

Two ways to claim the gap, and people muddle them constantly.

A power cap is the blunt one: you tell the driver "never draw more than X watts" and it enforces that by dropping clocks whenever the card threatens to exceed it. One command (nvidia-smi -pl 250 - my nvidia-smi guide covers it properly), works on Linux cleanly, reversible instantly, essentially impossible to make unstable. This is what I run on my own cards, because for a memory-bound workload the clocks it sacrifices weren't doing much anyway.

An undervolt is the finer instrument: you hold the clocks where they are but feed them less voltage, using a frequency-voltage curve in MSI Afterburner (Windows) or clock-offset tricks on Linux. Done well, it keeps full performance at lower power - better than a cap, on paper. The cost is that you're now testing silicon, not settings. Every die is different, an undervolt that's stable for an hour can crash overnight, and you have to soak-test it for days rather than minutes. Back when a percent of efficiency was money I ran curves on everything; these days I mostly can't be bothered, and the cap gets me most of the benefit for none of the babysitting. That's a preference, not a law.

Why workload changes everything

The reason there's no one right answer is that "GPU load" isn't one thing. Gaming is compute-and-raster bound: it will happily use every watt you allow and give you frames for them, so aggressive caps really do cost performance. LLM inference is bandwidth-bound: past the point where memory is saturated, extra watts buy almost no tokens - that's my 108W screenshot. Model loading and prompt processing spike compute harder than steady generation does, so a card that averages low will still have moments near its cap. And training or image generation sit somewhere between, hungrier than inference by a distance.

Which is why the advice is boring: the numbers that matter are the ones you measure on your workload. Published caps - including mine - are starting points from someone else's bench.

How to test a card properly

The method is the same one I use for benchmarking serving stacks, scaled down. Change one thing. Measure. Write it down.

1. Get a baseline. Your real workload (your usual model, your usual context), stock settings, and two numbers: performance (tokens per second, or frames, or whatever your work produces) and wall power if you can meter it, reported power if you can't. A £15 plug-through meter tells the truth; the driver's number is close enough to start.
2. Step the cap down. Drop the power limit 25W, re-run the same job, record both numbers again. Keep going. What you're looking for is the knee: performance holds nearly flat for several steps, then starts dropping in earnest.
3. Back off one step from the knee and soak it - run the real workload for a day before you trust it. Caps rarely misbehave, but thermals and boost behaviour shift over hours, not minutes.
4. Only then consider an undervolt, if the remaining watts are worth chasing on your electricity price. Same loop, smaller steps, much longer soaks, and expect to reach a setting that fails a week in. That's not you doing it wrong; that's silicon.

The whole exercise is an evening plus patience, and it's the single highest-value test almost nobody runs on hardware they leave on 24/7.

Where to start on your card

The nvidia-smi guide has per-card starting caps with the sourcing stated (my 4090 first-hand, the rest collated from owner reports), and every card page in the GPU comparison carries the settings people run. For rough shape: 30-series cards tend to give up a quarter of their budget for a few percent of inference speed, and the big Ada and Blackwell cards give up more still, because their stock limits are set for workloads most of us aren't running.

What I haven't done yet - and want to - is run the full cap sweep on the 48GB card with proper tokens-per-second numbers at each step, the way I once did for a very different workload. The 108W observation says the knee is far lower than I'd assumed, and I'd like to know exactly where before the second card doubles the stakes. When I've got that data it'll go here, and I'll find out whether the old intuitions still hold or whether this workload has new surprises in it. I half suspect the latter.