New Budget local LLM card: 5060ti with 16GB VRAM, check out its LLM Inference Speed
16GB VRAM is what is needed at the minimum if you don’t want to run the crap tear local LLMs, with 16GB VRAM you can easily run 14B and 24B parameters models in q4/q6 quant thus having a local LLM that is usable. NVIDIA’s latest offering 5060ti is going to be just that, featuring 16GB VRAM and 448GB/s… Read More »
