I am considering building a midlevel workstation for casual experiments with LLMs and occasional gaming. For GPU I am confused between used 3090/3080 vs new 4070/5070 12GB. 3090 has 24GB RAM and is available in 55-60k on some websites.
I will be pairing it with Ryzen 7 5800X and 64GB 3200Mhz RAM.
From what I have understood, VRAM is what matters most for LLM applications and 3090 also has much higher memory bus bandwidth(384bits) as compared to newer cards.
Also is there any real benefit of having dual GPU setup for LLM use case? I find contradicting opinions on it.
The 3090 will be better if you’re able to get it for a lower price cause you will be able to run larger models in its higher vram buffer.
Yeah, 3090 is the best easily attainable GPU for this.
The rumours of super series have begun with higher vram but those like a half year wait
Btw 3090 is 50-60k new? Where?
He meant used. They normally go for 55-50k, normally.
1 Like
Yeah currently, a used 3090 is the best bang for the buck for running local AI.
If you can increase the budget then a used 4090 < than 1 lac is a sweet deal.
But finding both of them is extremely hard. So might have to wait for a 5070 Ti Super with 24GB RAM.
Hey, Did you purchase one. I am also looking for same but does it really safe to buy used ones what is they don’t work or something goes wrong.
No, it went out of stock pretty quickly.
I was also very unsure of safety but I checked from different sources computify.in sounded legit to me. Also talked to their Chat support.
Recently found this youtube channel - https://www.youtube.com/@AZisk
He does lots of comparisons of various GPUs, Mac minis, etc for LLMs. You’ll definitely find a video relevant to you.
1 Like
Buy new 5070 ti 16gb.. i wouldn’t suggest buying 4090, way too overpriced.. while you can buy 3 5070ti with the cost of one 4090.. which means 3*16gb= 48gb vram when combined. If workload is single gpu only then go for 5090 32gb.. more vram is better
Yeah I have settled on 5070Ti for now.
1 Like