Hardware Setup for AI and ML research

doomboom

doomboom

Beginner
Hey FMs,

We are intereted to foray into AI and reinforced machine learning domain. Were wondering if any of you gurus can direct us towards scalable hardware which can be used to host and run various reinforced machine learning programs. We intent to start small but if things go our way, we should have the capability to scale it up. We've been hearing mixed responses as using retail GPUs vs using specific GPU clusters been developed by Nvidia. We intent to do pattern determination and lingustics work in RML.

Any help towards this will help. Finances are not a concern.

Thank you in advance sirs. :)
 
ibose said:
As a startup, won't using one of the cloud based offerings be more feasible?
Click to expand...
Yes its more economical for sure, but there are a lot of issues related to trainig data theft, model copying etc. Then there are regulations with respect of training data handling etc. Its better to start with own setup is what we've concluded as a team.
 
doomboom said:
Yes its more economical for sure, but there are a lot of issues related to trainig data theft, model copying etc. Then there are regulations with respect of training data handling etc. Its better to start with own setup is what we've concluded as a team.
Click to expand...
I would suggest exploring the cloud solutions a bit more on the issues you mentioned. Talk to some of the vendors before discounting it altogether. If you have already done that due diligence then please ignore.
 
  • Like
Reactions: doomboom
doomboom said:
Yes its more economical for sure, but there are a lot of issues related to trainig data theft, model copying etc. Then there are regulations with respect of training data handling etc. Its better to start with own setup is what we've concluded as a team.
Click to expand...
Using cloud services may or maynot be economical.
If your business is growing ( I am talking about AWS/Azure with IaaS/PaaS)
  • linearly then using cloud services for few years(less than 3) will be cheaper
  • exponentially then using cloud services is a bad idea it will be costly
Now to your requirement if you are going to use Llama2/Stable diffusion/inference tasks etc look for NVIDIA cards

Turing generation
  • T4

Ampere
  • A10
  • A16
  • A30
  • A40
  • A100
If budget is a constraint and you want the RTX cards then look for (used or new)
  • 3090
  • 4090

If you are unsure about your algorithm's performance then start with a single 3090 (you can easily get your hands for trial).

If buying new start with A10 24GB(retails ~ 3.5lacs INR)
If budget no constraint A100 40GB (retails ~12 lacs INR)
 
  • Like
Reactions: doomboom
I would always prefer personal hardware as one has full control over everything and can be done with used hardware anyway want.
Rtx 3090 as of now used can be had for 50-55k and sometimes have seen rtx A5000 also but person was asking 2lac.....which i think is quite higher. But i wud also go for 3090 and if required can use nvlink with 2 3090.
 
This topic or OP might be helpful as well

2x3080 ti or 3090 for ML?

Hi I have 2x3080ti + 3090. The 3080ti's are connected to the motherboard through risers. So they run at x1. 3090 is in x16 slot. My question is, if i get an additional 3090, would it be better than 2x3080ti + 3090? That would take less power, and the additional card would fit into my cabinet...
techenclave.com techenclave.com
 
  • Like
Reactions: rockyo27
Top Bottom