NVIDIA RTX 5000 series CUDA cores and Memory bandwidth comparison table

In this another lets compare series we are going to compare the upcoming (in two weeks) NVIDIA (RTX 5090, 5080 , 5070ti and 5070) with older generation based on the CUDA cores and memory bandwidth alone, both of these metrics are required to make a buying decision if want to run AI locally on your computer. Nvidia’s CUDA cores and VRAM is the gold in the current AI industry, these determine whether your PC will be handle the tasks like local AI, image generation using Flux etc. As per the state of current models you will be needing a lot more VRAM in the days ahead.

Cuda cores and bandwidth comparison for NVIDIA GPUs:

Which one to buy?

Which NVIDIA 5000 series GPU to buy if you wish to run a Qwen2.5-Coder-32B q4 model or similar ones locally?

It is suggested that you buy only 16GB VRAM and above graphics card as per my experience. For LLM’s alone all future purchases if you are purchasing new should be above 16GB. For eg. I am able to run the above model locally using a 3080ti that only has 12gb VRAM and allowing the q4 model to spill over to memory that slows the token generation to about 3-4 tokens/second.

Also the upcoming GPUs have hardware decoders for 4 bits built into tensor cores that should significantly boost the local run of quantized AI models.