New GPU!

I’ve been saving up for a year and change for a GPU upgrade, and my local Microcenter finally had 5090s in stock. So I bought one. It is a CHONK – I’m getting full “you vs. the guy she tells you not to worry about” vibes:

Specifically, I’ve upgraded from a PNY RTX 3090 XLR8 Gaming EPIC-X RGB (what a mouthful) to a PNY RTX 5090 OC. Some of the size difference is due to “reference” vs “custom” PCB and cooler design; the 3090 is a reference design while the 5090 is a custom board and cooler. And it needs the cooler – the card now pulls up to 1.7x more electricity at up to 600W (no wonder Nvidia has problems with cables melting)! Regardless, I watercool the GPU in my system, so the size of the cooler shouldn’t matter (that assumption will come back to bite me in the ass).

The neat thing about these graphics cards with 24GB+ VRAM is that they can run super-capable LLMs locally. Models like Qwen 3 30B A3B, Qwen 3 32B, and Gemma 3 27B match or outperform proprietary models via paid API; the 5090’s token evaluation rate (which impacts experienced user latency) is also roughly equivalent to that experience via paid API. So I now have a local, private LLM instance that is functionally equivalent to GPT-4o or Claude 3.7 Sonnet! Granted, I’d have to have send on the ballpark of a quarter-million prompts to my self-hosted model to recoup the cost of the GPU instead of just using a paid API… but the upgrade was not intended to be a “cost saving” measure.

60% Performance Improvement

After testing, I can confirm that the 5090 is faster. 🚫💩🕵️‍♂️ – it’s 2 architecture generations (Ampere -> Ada -> Blackwell) and 5 years (2020 -> 2025) / card generations (30xx -> 30xx Ti -> 40xx -> 40xx Ti -> 40xx Super -> 50xx) newer.

I tested with aidatools/ollama-benchmark, a very convenient command line benchmark based on Ollama that tests throughput over a variety of LLM and VLM architectures and sizes.

LLM Benchmark Results: RTX 3090 vs RTX 5090

The benchmark indicates the 5090 has roughly 60% performance improvement over the 3090; the increases are larger as model size (parameters) increase. As the models used get even larger, I anticipate an even greater performance lift for the 5090, as a result of more tensor cores, more efficient tensor cores, and higher memory bandwidth; as the total utilization of the available compute/memory bandwidth approach 100%, the 5090 should increase its lead over the 3090.

System Specs

My PC is in an Alphacool 4u server chassis that supports water cooling; this lets me keep noise down, have quite good cooling performance, and I can stack my computer in a server rack with the rest of my homelab gear. For the non-nerds, this means that my computer is functionally “on its side” whereas a normal PC stands tall.

You may notice the graphics card (top right, with “CORE” in shiny silver) is horizontal. This is because the 5090 is too big for the chassis, especially with the protrusion on the water block for connecting the water lines. So I had to hack the case with a “vertical GPU mount”; since the chassis is “on its side” relative to a normal computer, this means that the “vertical” mount lets me hold the card horizontally.

Blogumentation

Here are my notes on configuring my system for the experiment:

1. Install WSL (Windows Subsystem for Linux)

• Install WSL, reboot
• Install distro (I use Debian)
• Configure distro (Create user account, etc.)

2. Install dependencies (in WSL)

• Install standard dependencies

  apt install curl wget git software-properties-common build-essential

• CUDA for WSL

  • Install nvidia cuda-toolkit
    ⚠️ DO NOT INSTALL THE DRIVER ON WSL; ONLY INSTALL THE CUDA-TOOLKIT ⚠️

  • Check GPU device post-installation using the compiled deviceQuery binary

    apt install cmake
    mkdir build && cd build
    cmake ..
    make -j
    ./deviceQuery

• Install ollama

  curl -fsSL https://ollama.com/install.sh | sh

• Install pipx

  apt install pipx

• Install uv

  curl -LsSf https://astral.sh/uv/install.sh | sh

• Install Docker Desktop

• Install nvidia container toolkit - I did this, but I’m not sure whether it is necessary for this particular experiment

3. Install aidatools/ollama-benchmark

pipx install llm-benchmark

4. Run tests

• Close all apps

• Disable internet

• Run:

  llm_benchmark run

Cite this article

If you would like to reference this article, please consider citing it as:

Graber, A. H. (2025, May 25). New GPU!. AI/MLbling-About. https://aimlbling-about.ninerealmlabs.com/blog/gpu-testing/

Or with BibTeX:

@online{graber2025_gputesting,
  author = {Graber, },
  title = {New GPU!},
  year = {2025},
  date = {2025-05-25},
  url = {https://aimlbling-about.ninerealmlabs.com/blog/gpu-testing/},
  urldate = {2026-08-16},
  note = {Blog post}
}