CPU/Mobo Virtualization Lab

vivek.krishnan

vivek.krishnan

BLR~ZRS-TX-1-MX
Skilled
So, I am building a dual socket Xeon build for Hyper-V. This will be used for testing and checking performance of virtual machines vs bare metal machines.

We will be going with the following for now :

1. Dual E5 2670 V1 Xeons. These are pretty cheap right now, about 75 USD. Importing these should come to around 12-15k for a set.
2. Dual socket Supermicro motherboards. These are about 30K for the board, and are pretty decent. Additional heatsinks would be needed.
3. DDR3 ECC RAM - 10K for 16GB of RAM. Crucial.
4. Drives. We would be going all SSD for everything. 850 EVO 250GB for about 7.5K each. RAID1 for the Hypervisor and base OS. RAID10 for data?
5 Cabinet and PSU - Not yet decided.

Suggestions?
 
There are sooo many possibilities... is it for personal use or work related? Any plans for iSCSI san/nas env? I would seperate storage from the compute. boot the machine using something like 8G usb drive or network boot. for RAID; something better than raid1/raid10 with increasing the spindle count so that read perf is better.
 
Will be at work. We will be going for NAS, which would have iSCSI support - could be freenas or synology - but after a couple of months. SAN would be a no.

For us, workloads are more like 50-50 read writes for files and more or less equal for SQL server. We are running with a dedicated RAID 10 for FS and DB with SAS 10K, while we earlier tested with 250GB Samsung EVOs in RAID10.

We wont be using a bootable Hyper V or ESXi pen drive, but mostly a pair of SSDs in RAID1. These would also contain the images for booting the VMs. Another array would be for db and FS.

The Xeons came to 12K for a set shipped to India. Got the details from ServeTheHome forums.

The board cost us 32K+ taxes. http://www.supermicro.com/products/motherboard/Xeon/C600/X9DRD-iF.cfm
 
Last edited:
tommy_vercetti said:
Can you share more on boot the machine use USB drive, what would you be using?
Click to expand...

You want to know how to boot with a USB drive, or which OS to use, or what pen drive to use.

Googling will give an answer to the first one. OS to use is your choice - ESXi/Hyper V/others...

Pen drive, I think should be something along the lines of Corsair Voyager, which would have good read/write speeds.

I would have done this, but I want WS 2012 R2 Standard for the base OS for administrative capabilities. Maybe Hyper V in the future.[DOUBLEPOST=1459430730][/DOUBLEPOST]Heatsinks had been ordered from Astric -SNK-P0048AP4. They have arrived as well.

Awaiting the board....
 
Last edited:
vivek.krishnan said:
1. Dual E5 2670 V1 Xeons. These are pretty cheap right now, about 75 USD. Importing these should come to around 12-15k for a set.
Click to expand...
Am i reading something wrong here.Are they refurbished?
I remember a seller selling for 32K on ebay.in INTEL XEON E5-2670
Even Amaon is selling for 400$ Intel E5-2670
Buying from the US would be a gamble in itself unless its a trusted seller.
Desktop processors can wait but not for servers.

We have our HyperV setups on 2650 and 2667

I feel 16 GB for Hyper V would be very less and because the VM OS itself will consume space on top of bulky WS 2012
 
try to get lsi megaraid, it supports upto 8 drives and things line raid 50 raid 60 etc... along with hotswap drives, battery backed cache.
 
Arv90 said:
Am i reading something wrong here.Are they refurbished?
I remember a seller selling for 32K on ebay.in INTEL XEON E5-2670
Even Amaon is selling for 400$ Intel E5-2670
Buying from the US would be a gamble in itself unless its a trusted seller.
Desktop processors can wait but not for servers.

We have our HyperV setups on 2650 and 2667

I feel 16 GB for Hyper V would be very less and because the VM OS itself will consume space on top of bulky WS 2012
Click to expand...

As I said, this is for a trial, testing - not production ATM. Secondly, the seller is trusted, yes. Finally, we have several bare metal servers, and would like to evaluate the possibility of virtualisation.

I don't think there is anything to refurbish in a CPU except check if it is working :confused:[DOUBLEPOST=1459445316][/DOUBLEPOST]
booo said:
try to get lsi megaraid, it supports upto 8 drives and things line raid 50 raid 60 etc... along with hotswap drives, battery backed cache.
Click to expand...

Not planning for hot swap. Have BBU RAID controllers as well. But as I said, we are going to try with onboard stuff first.
 
Although this is very simple usage but I tried a had go with both Hyper V on server 2008 and ESXi on my home server. It was a old Dell Poweredge 2950 with a single E5420 and Perc 6i raid card. Was running 3 VMs 2 running different flavours of Linux and one Windows 7. This is just my play around test system at home though so no enterprise grade budget or usage.

I was having a go at seeing the performance of hosting website and running a minecraft gameserver at home. How is your test project coming along. What kind of use you plan on using your system for.
 
booo said:
hmm, have you considered openstack/ovm?
Click to expand...

As of now, no. For the future, we will be testing all sorts of stuff on that - including Ws2016 when it releases and open stack. For now, we are going to simulate a test to check VM performance of our existing setup, HA. Then we would be testing anti ransomware defenses. These should be done in about 2-3 weeks.
 
ronnie_gogs said:
Although this is very simple usage but I tried a had go with both Hyper V on server 2008 and ESXi on my home server. It was a old Dell Poweredge 2950 with a single E5420 and Perc 6i raid card. Was running 3 VMs 2 running different flavours of Linux and one Windows 7. This is just my play around test system at home though so no enterprise grade budget or usage.

I was having a go at seeing the performance of hosting website and running a minecraft gameserver at home. How is your test project coming along. What kind of use you plan on using your system for.
Click to expand...

I actually ordered the same system, but it cancelled due to OOS. Then I hit upon doing this, which would be running nearly current generation systems plus be able to run VMs too - and my boss too liked the idea.

As of now, only heatsink and CPUs are with me. Will be initially testing with 16GB of DDR3 ECC RAM, then move to full density RAM. Should be using only SSDs for deployment ideally.
 
ronnie_gogs said:
Although this is very simple usage but I tried a had go with both Hyper V on server 2008 and ESXi on my home server. It was a old Dell Poweredge 2950 with a single E5420 and Perc 6i raid card. Was running 3 VMs 2 running different flavours of Linux and one Windows 7. This is just my play around test system at home though so no enterprise grade budget or usage.

I was having a go at seeing the performance of hosting website and running a minecraft gameserver at home. How is your test project coming along. What kind of use you plan on using your system for.
Click to expand...

I actually ordered the same system, but it got cancelled due to OOS. Then I hit upon doing this, which would be running nearly current generation systems plus be able to run VMs too - and my boss too liked the idea.

As of now, only heatsink and CPUs are with me. Will be initially testing with 16GB of DDR3 ECC RAM, then move to full density RAM. Should be using only SSDs for deployment ideally.

Firstly, we need to assess the issues of migrating to VM. Almost all the softwares we have are based on IIS, ASP .NET, SQL - how they will handle being in a VM.
 
I am also considering to play with this new consumer grade option which came into my consideration after I saw a linustechtips video. unRaid https://lime-technology.com/. I am sure it wont be suited for a enterprise setup such as yours.

Next plan is to play with cluster computers. My friend who works on a 3000 core cluster for his work is helping me on a making a SBC cluster like banana pi or beaglebone black. Will start with something like 5 computer cluster for start.
 
I would pass over unRAID. It has some limitations which are not really great. Similar to Nexenta Stor (Cant recall the actual name)

Parallel computers have been around a while. However, I dont think it will stick unless you have scale. Had looked into it sometime back, will update in sometime.
 
Motherboard will be in my office by tomorrow.
Will be using a VIP 600W Gold PSU to power the system for now, albeit temporarily.
Will be borrowing a R9 270X from a friend if needed.
16GB of DDR3 ECC RAM will be used initially - 8GB per channel to test only.
A pair of RAID1 Samsung 850 EVOs will power the storage subsystem.
Will buy a case once I see what I need!
 
Well depends on the application. Rendering applications or computational applications run much better on a graphics processor. And I find OpenCL much better than CUDA for non gaming applications.

Heck I used to run my own Mining computer with 6 gfx card. 3 7970 and 3 7950 :p. Back when I was into cryptocurrency mining .. haha
 
OK, so got the board today morning. A quick visit to Lamington to buy the other stuff was planned.


Was unable to pickup UDIMMs - had to settle for Kingston 1.35V LRDIMMs running at 1600 Mhz. Cost 7.5K for 16GB, got 4. Will be running memtest overnight to check stability.

Chassis turned out to be a major issue with Prime suggesting to go with NZXT cabby for 18K + 12.5K for the Corsair RM1000. Eventually went for Chenbro RM23608 from Arihant Info Solutions (opposite Naaz Cinema - a haven for assembled servers, if I may so), cost me about 27K. 2U rack server, with 8x3.5" hotswap bays. Comes with a 80+ 500W PSU standard. Saw like about 10-15 of these. He also had a pleortha of server stuff and NAS stuff as well. You may want to talk to Vinayak, Mukhesh and Abhishek - they know their stuff.

Ditched the idea to buy 8x SSDs, went with 4 SSDs for boot + VM and 4x Seagate standard HDDs for data storage. Configured in RAID1, RAID1 and RAID10 array.

Total cost so far - 1.5 lacs. Overshot a bit, but a 2U rack! Pictures will follow.[DOUBLEPOST=1459986981][/DOUBLEPOST]Total BOM so far :

2x E5 2670 V1's (from USA) - 12K
1x Supermicro X9-DRD-iF (from Netweb)- 33.5K
4x 16GB Kingston RDIMMs @ 1600 MHz, 1.35V (Prime) - 28.5K
Chenbro RM23608 2U chassis with 500W PSU (Arihant, billing from Prime)- 27K
4x Samsung 850 EVO's (Prime) - 28.5K
4x Seagate 7200 RPM 1TB HDDs (Prime) - 15.5K
2x Supermicro Heatsink - SNK-P0048AP4 (Astric, Kolkata) - 5K

Total cost - 1.5 lacs.

Forgot to take pics last night, and the chassis is already rack mounted. Ill try today.
 
Last edited:
Back
Top