Budget 71-90K Building a server (first-timer!)

theGloaming

Disciple
I've decided to take the plunge and build myself a server. I've done some research, ordered a few parts, and need some help with the pending parts, configuration, and setup. (Though in this thread I'd like to focus just on getting the complete hardware defined. The setup I'll post in another thread once I am ready to start assembling).

What I Intend to do with it
  • Run it as a headless 24x7 server for learning, experimenting, and developing. Will move some applications to production on the same or new setup (as needed).
  • Run a number of VMs on it, as well as containers e.g. for home automation, security, software development, data analysis, email server, etc. Thus I need many cores.
  • Run FreeNAS on it in a VM using KVM (essentially to experiment if I like it and it gives me value. If so, I intend to move it to production later on this same server or a separate physical setup on bare metal). Thus, I need ECC RAM and PCI-passthrough.
  • I have no intention to use it for gaming.
What I have already purchased
  • Motherboard - Supermicro X9SRL-F-O
  • CPU Cooling - Noctua NH-U12DXi4
  • RAM - Samsung 16GB (x2) DDR3 1600 ECC Registered
  • SSDs - Samsung 850 EVO - 120GB (x2)
What I must still purchase
  • CPU - (likely) Intel Xeon E5-2670 / 2660
  • HDDs - 2x4TB + 6x2TB OR 2x4TB + 6x4TB (likely WD Red. 2 or 4TB depending on cost)
  • Case - to be decided
  • Power Supply - to be decided
  • HBA - (likely) SAS9211-8I

Help Needed
  1. Would you suggest differently on the things I must purchase?
  2. Would you have any specific suggestions on the aspects I still need to decide (Case / Power Supply)?
  3. Thoughts on where I might get these within India?

Thanks much!
 
[And I just noticed this template, and here it is filled in]

  1. What is your budget?
    • 80K
  2. What is your existing hardware configuration (component name - component brand and model)
    • Motherboard - Supermicro X9SRL-F-O
    • GPU - None
    • RAM - Samsung 2X16GB DDR3 ECC Registered RAM 1600
    • Monitor - None
    • CPU Cooling - Noctua NH-U12 DXi4
    • SSDs - Samsung 850 EVO 120GB (x2)
  3. Which hardware will you be keeping (component name - component brand and model)
    • All of the above
  4. Which hardware component are you looking to buy (component name). If you have already decided on a configuration then please mention the (component brand and model) as well, this will help us in fine tuning your requirement.
    • CPU - Intel Xeon 2670 (or perhaps 2660 / 2650 depending on the cost). Can be v1 or v2.
    • HDDs - 2x4TB (for server) + 6x2TB (for NAS) OR 2x4TB (for server) + 6x4TB (for NAS). Likely WD Red.
    • Case - Quiet case, which can house at least 8 3.5" + 2 2.5" drives. Preference is Fractal Design R5 (but this is not available in India, as isn't be quiet!). Currently contemplating NZXT H440 or SilverStone Raven RV03.
    • PSU - to be decided after I choose the case
    • HBA - SAS/SATA expander. Likely LSI Logic SAS9211-8I
    • RAID controller - Might be needed for the 2 SSDs + 2 server drives
    • Cables and such
  5. Is this going to be your final configuration or you would be adding/upgrading a component in near future. If yes then please mention when and which component
    • Don't intend to make major changes in the next few months.
    • Only might start with 2x4TB HDD's and buy remaining 6 HDD's over the next few months and then install them
    • Might (might) invest in a GPU in case I start running machine learning on this setup
  6. Where will you buy this hardware? (Online/City/TE Dealer)
    • Pune (but can travel to Mumbai if needed)
    • Open to online purchase
  7. Would you consider buying a second hand hardware from the TE market
    • No
  8. What is your intended use for this PC/hardware
    • Home Server for home automation, software development, data analytics, email server, web server etc.
    • Hosting a number of VMs and Containers
    • NAS (FreeNAS)
    • No Gaming!
  9. Do you have any brand preference or dislike? Please name them and the reason for your preference/dislike.
    • I'd prefer a Fractal Design R5 case or a be quiet! 800 / 900 - but don't find them in India at reasonable cost.
  10. If you will be playing games then which type of games will you be playing?
    • NA - no gaming
  11. What is your preferred monitor resolution for gaming and normal usage
    1. Desktop - Full HD (1920*1080) But normally will use this server as headless
  12. Are you looking to overclock?
    • No
  13. Which operating system do you intend to use with this configuration?
    • Linux - Ubuntu Server
    • FreeBSD (in VM)
 
For an entry level setup, this is a great setup. I have the dual socket setup, and it's not in use as of now, but will be in a few weeks as a HA Hyper V setup.

The only negative for this CPU/board combo is the power draw which is huge.

Coming to the case, I would suggest to skip the cases you have mentioned, and opt for a server case, which will allow you to have hot swap.

Additionally, you can also run FreeNAS on the physical server and run the others as VMs on top. @cyberwarfare will give you more ideas

However, if you want to get the best, I would suggest to run a more mature production hypervisor such as ESXi or HyperV or etc...

This will allow you to have more options, including running your router as a part of the virtual infrastructure.

Finally, if you have not finalised on this, would suggest to pickup something which will be less power hungry.
 
For an entry level setup, this is a great setup. I have the dual socket setup, and it's not in use as of now, but will be in a few weeks as a HA Hyper V setup.

The only negative for this CPU/board combo is the power draw which is huge.

Coming to the case, I would suggest to skip the cases you have mentioned, and opt for a server case, which will allow you to have hot swap.

Additionally, you can also run FreeNAS on the physical server and run the others as VMs on top. @cyberwarfare will give you more ideas

However, if you want to get the best, I would suggest to run a more mature production hypervisor such as ESXi or HyperV or etc...

This will allow you to have more options, including running your router as a part of the virtual infrastructure.

Finally, if you have not finalised on this, would suggest to pickup something which will be less power hungry.

@vivek.krishnan - thanks for your insights! It does give me food for thought, and I'll also be happy to hear @cyberwarfare's thoughts.

I will be happy to use a server case, but have not been able to find any (which is not crazy expensive). I'd love hot-swap bays too.
Would you have an idea on which are the better options, at an entry level price point?

In terms of finalization, I still have to buy the CPU, HDDs, case and power supply; the motherboard and RAM though are already in my possession.

=======

Let me give a bit more background on how I came to these choices - that should help choose some directions going forward. [Please do note that I am a newbie and a lot of this information has been gleaned in the last week and a half so I might have some concepts wrong. I'm happy to be corrected though!]
I fear this might be a long post, so at first TLDR;
I want to keep the system OS as Linux, and run VMs/apps/... on this as I learn and experiment (running the router on this is also one of the thoughts to experiment with). Eventually as the need for these apps/solutions gets established, I will likely move specific applications to production grade and separate physical environments (with different OS's and configs), but for now going that way would be overkill. Eventually, I might even move a number of these things to AWS or Google Cloud (but not all!).
FreeNAS itself is an experiment to figure out if it makes sense to have it running at home, and if so what kind of things I will store, what will be my data storage needs, etc. Initally I will run it on KVM. These aspects lead to a choice of ECC RAM and Xeon CPUs (more below in the longer version). I opted against a dual CPU setup for power draw reasons, and hope that a single CPU server will not bring on a crazy power bill.

The Longer Version
The idea of this setup started with my desire to get my hands dirty on Linux. I want to learn much more of Linux both on server and PC (am also moving an older laptop to Ubuntu). Eventually, this might lead to more specialized needs and setups, but for now the idea is play/experiment/learn/mess-up-and-fix (even FreeNAS at the moment is in an experimentation mode - more on this later). This is why I like to go in with an entry level system - as I move to more production for certain applications, I will likely need more specialized, mature, and expensive systems. But at that time it will all be easy, as I will then know the value of these applications to me, and what kind of power/storage I will need to meet my needs.
Linux also allows me to run VMs on KVM (also a type 1 hypervisor), as well as learn about deploying in containers. And allows me to form a solid basis before moving on to learn things in the cloud. And probably a whole bunch of other things which I don't know now.
These also then are reasons is also why I am not running VMs on FreeNAS but rather on the Linux server, and not going for other hypervisors.
Thus, I want to keep Linux running on the hardware with no other hypervisor in between (e.g ESXi or HyperV). Where I need a type-1 hypervisor, I'll go with KVM.

FreeNAS itself is an experiment. It seems to be a good use case to run on my server. I'll probably move my laptops backups and photos, videos, and music to this (after re-ripping the music and movies to lossless codecs). This will let me figure out what kind of transcoding power I will need. I'll likely also run Plex and MythTV on this. And well, whatever else comes my way that seems useful.
Once I've seen that it makes sense to have FreeNAS at home, then I'll likely move it to a stand-alone bare metal setup down the road (from all my reading on FreeNAS it is not recommended to run it virtualized and thus if I choose to stick with it, I'd rather go for the safer approach). Else, it will be consigned to the dump and I'll move on to something else.
The need to FreeNAS (including ZFS) necessitates ECC RAM - and thus all Intel Core CPUs are out. Running FreeNAS as a VM needs VT-d / PCI Passthrough. I have read the Ryzen CPUs can have issues with this. This then essentially leaves Xeon as the best feasible option.

Among the Xeon chips, I figured that I I'll go with a 26xx chip rather than a 16xx chip as I get more cores, and if ever needed I can use the 26xx chip in a dual CPU config. More cores help as I foresee running multiple VMs and I'd rather have a few more rather than less.
I'm sticking with the v1/v2 versions as they are less expensive than the v3/v4 versions (I'm trying to see if I know someone coming to India from abroad in the coming weeks who can bring one down - they are so much cheaper over there).

In terms of memory / storage (this is still an evolving structure):
  • I'll have 2 SSDs running the server OS (in a mirror or RAID1)
  • I'll have 2 HDDs holding the server storage - likely 2x4TB (also in a mirror or RAID1).
  • I'll have 3-6 HDDs for FreeNAS - unsure if I will use 2TB or 4TB HDDs for this. Thus, I might be able to go for case with less than 8 HDD bays (but at least 6 dedicated HDD bays - hot swappable would be awesome but not mandatory in the server based FreeNAS). In production though I intend to have vdevs of 6x4TB in RAID-Z2 (this seems to be a good option from my rather limited reading). Here (in production FreeNAS) I would want hot swappable drive bays.
Hope that made sense and is not too disjointed - kind of putting together thoughts of the last week and a half.
 
Im sorry i couldnt reply sooner.

I recently (just about a year ago) built a new NAS. The old one was running an older amd opteron. The new one is based on a supermicro x11ssm-f and intel xeon 1230v5 with 16gb of ecc RAM.

Ive been using FreeNAS for about 5 years now, and i must say that while its finicky about the HW its runs on, its rock solid. I run a VM on my box and it handles it just fine.

If you have any questions, do PM me and ill be glad to help you anytime :)
 
Im sorry i couldnt reply sooner.

I recently (just about a year ago) built a new NAS. The old one was running an older amd opteron. The new one is based on a supermicro x11ssm-f and intel xeon 1230v5 with 16gb of ecc RAM.

Ive been using FreeNAS for about 5 years now, and i must say that while its finicky about the HW its runs on, its rock solid. I run a VM on my box and it handles it just fine.

If you have any questions, do PM me and ill be glad to help you anytime :)

I'm happy that you replied and that we can discuss this - time (thus far) is still on my side!

That NAS of yours looks like a sweet system! If / when I do move my NAS away from the experimental server setup to a standalone production system, I might look at a configuration like yours! I'm definitely going to come back to you with many FreeNAS questions - once I have all my server parts and once I have it installed and running.

Would have some insight on where one can get Xeon's in India? I am looking for an E5-2670 (or perhaps a 2660 or 2650), new or used, but no much luck in finding it here at a reasonable cost. The best options I see are to order it over amazon (USA), or ebay (from USA, China, or Hong Kong).

Would you know how best to go about this, viz, judging who is a reliable seller? And also figuring out customs costs and logistics (I worry that the CPU might never make it past customs).
 
I'm happy that you replied and that we can discuss this - time (thus far) is still on my side!

That NAS of yours looks like a sweet system! If / when I do move my NAS away from the experimental server setup to a standalone production system, I might look at a configuration like yours! I'm definitely going to come back to you with many FreeNAS questions - once I have all my server parts and once I have it installed and running.

Would have some insight on where one can get Xeon's in India? I am looking for an E5-2670 (or perhaps a 2660 or 2650), new or used, but no much luck in finding it here at a reasonable cost. The best options I see are to order it over amazon (USA), or ebay (from USA, China, or Hong Kong).

Would you know how best to go about this, viz, judging who is a reliable seller? And also figuring out customs costs and logistics (I worry that the CPU might never make it past customs).

Ill be glad to help you wherever i can :)

As for the components, i picked up my components from a local distributor who goes by the name Arihant Infosystems. The prices are on the higher side (when compared to the US rates) but are way cheaper than other local distributors.
Eg. Boston quoted ~26k for the Supermicro x11SSM-f whereas i got it for less than 20k from Arihant. He stocks only new components and will organize anything else that you may need, if you want.
 
+1 to AIS. For servers, he is my goto guy. Pulled out this email from him.

Intel Xeon Processors for Servers & Workstation #

E5 Series V4
2640 v4 (10 Cores / 2.1 GHz)
2637 v4 (4 Cores / 3.5 Ghz)
2630 v4 (10 Cores / 2.2 Ghz)
2620 v4 (8 Cores / 2.1 Ghz)
2609 v4 (8 Cores / 1.7 Ghz)

E5 Series V3
2699 v3 (18 Cores/ 2.3 Ghz)
2697 v3 (14 Cores/ 2.6 Ghz)
2680 v3 (12 Cores / 2.5 Ghz)
2667 v3 (8 Cores / 3.2 Ghz)
2643 v3 (6 Cores / 3.4 Ghz)
2637 v3 (4 Cores/ 3.5 Ghz)
2630 v3 (8 Cores / 2.4Ghz)
2620 v3 (6 Cores / 2.4 Ghz)
2609 v3 (6 Cores / 1.9 Ghz)

E5 Series v1
2620 (6 Cores / 2 Ghz)
2670 (8 Cores / 2.6 Ghz)
2660 (8 Cores / 2.2 Ghz)

Xeon 5650 / 5645 / 5640 / 5620

Xeon 5503/5504/5506/5507/5520

Xeon 5450/5430/5420/5410/ 5405


Other Processors can be arranged on B2B basis....


Contact #
Arihant Info Solutions,
8/9 A Lamington Chambers, 394 Lamington Road,
1St Floor, Opp Imperial Cinema,
Mumbai - 400 004
Tel : +91-22-2382 2424

Contact # Abhishek / Shailesh / Mukesh
 
Last edited:
Ill be glad to help you wherever i can :)

As for the components, i picked up my components from a local distributor who goes by the name Arihant Infosystems. The prices are on the higher side (when compared to the US rates) but are way cheaper than other local distributors.
Eg. Boston quoted ~26k for the Supermicro x11SSM-f whereas i got it for less than 20k from Arihant. He stocks only new components and will organize anything else that you may need, if you want.

That sounds great - I prefer a local trusted dealer!

To confirm is this the Arihant you were talking of?
http://www.arihant.net.in/
 
Back
Top