Will AMD or Intel have unified memory in future similar to Apple for running LLMs locally?

What do you think is wise?

  • Go for the PC as its versatile

  • Go for Mac studio as its energy efficient and linux based OS so would be good for professional use

  • Wait for cheaper alternative from AMD or Intel

  • Nah! Pay the subscripton and avoid the hassle as this is the cheapest way out

Results are only viewable after voting.
ramsudharsan75

ramsudharsan75

Apprentice
Recently there has been a lot of buzz in the AI race. With recent launches like Qwen 2.5-coder models demanding around 20 GBs of VRAM and new MCP superpowers, I'm feeling the itch to run these models locally instead of paying subscription fees to services like Github Copilot or Cursor. These services also limit the number of tokens per request thus restricting their full potential.

I know you can use your API keys in extensions like Cline to unlock them, but that would mean you'll shell out a lot of bucks very quickly, hence this discussion. There are a couple of options, either buy a GPU to run them in Windows (5090 32GB) or buy a Mac Studio (36GB). In the former option, it will consume more power raising the electricity bills than the latter (but by how much?) and would need other components thus raising the setup cost but will be able to game on it if needed :).

One question keeps popping up though - will there be any Apple alternative that will have unified memory similar to it by AMD or Intel in the near future? It's already several years now but I'm a bit disappointed by the lack of hardware by non-Apple CPU manufacturers. Let me know your thoughts. Interesting times ahead for sure with the smaller AI models.
 
As mentioned AMD has already launched cpus with architecture similar to Apple unified memory.

You also have the option of running it on just CPU and RAM, of course it will be much slower than a dedicated GPU or any of the unified memory systems but if speed is not as much of a factor than it is the cheapest way to run large models locally.
 
MostlyHarmless said:
As mentioned AMD has already launched cpus with architecture similar to Apple unified memory.

You also have the option of running it on just CPU and RAM, of course it will be much slower than a dedicated GPU or any of the unified memory systems but if speed is not as much of a factor than it is the cheapest way to run large models locally.
Click to expand...
I have replied at least 3 times with similar replies in the past couple of months over here. Just seem to be going around in circles with all such similar new posts.

Off topic, nice to see JC in the profile pic. Reminds me that I need to get back to trying out RTX Remix with Deus Ex again.
 
  • Like
Reactions: MostlyHarmless
One thing you could do if electricity prices are low where you live and want to get close to 4090 performance which is I think similar to the M4 max in terms of AI performance, is build a dual 3090 machine with used 3090s of course and with an older 7002 series epyc processor, the whole system would cost about 2L I think, but it would be significantly cheaper than a mac studio and would probably outperform it in larger models, it is also scalable as you can add more gpus maybe 2 more 3090s and it would be able to run even a 70b model with 8 bit quantization.
 
ramsudharsan75 said:
will there be any Apple alternative that will have unified memory similar to it by AMD or Intel in the near future?
Click to expand...
Obviously, apple is the trend setter, it has the power to shape the industry, every new thing apple has done, good/bad, opens up a huge market for money making for the competitors.

I say there will be many non-Apple chips for this purpose.
 
Top Bottom