New home NAS and Hypervisor setup

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

MikeC

Member
Apr 27, 2013
59
11
8
UK
Thanks for the info guys. I always get a bit worried with electronics and wiring stuff myself. I understand the basics, but not very practical. I get a bit heavy handed with fiddly little things. Just out of interest how hot will that resistor get? And how does that wiring trick work without a resistor then? I don't like it when things go bang if I get them wrong.

I think getting a new case was a bit of overkill as I was annoyed at the time :rolleyes:

So found a few bits at QuietPC.co.uk that should do the trick I hope, and still let me use all the original bits and pieces.

AcoustiFan Anti-vibration Soft Silicone Fan Gaskets

FANMATE 2 Variable Fan Speed Controller

RC56 (7V) Noiseless Resistor Cable

RC100 (5V) Noiseless Resistor Cable

ZM-MC1 Fan Multi-Connector Cable

I thought that I'd get the gasket as well to stop the vibrations on the case. Lots of little wired up leads and an adjustable fan controller there as well. Too much choice for me at the moment to decide. Any suggestions or a good explanation of how the re-wiring works. I might feel brave enough to attempt it :p

Another thing that I need are some good Cat6 network leads. I see loads advertised everywhere. Does anyone know a good but cheap place in the UK to order from? I am looking mainly for 1m leads and want different colours too. Probably 10 x red, 10 x blue, and 5 x yellow. Got plenty of much longer grey ones but I want to start getting the gear in it's final position before I buy the hardware to make the stack of virtual host servers
The wiring trick works because the difference between 12v and 5v is 7v, so +12 - +5 = 7 with the 12v wire being "more positive" than the 5v one. I can recommend HDMI Cables, Optical & USB Cables, Network & Audio Cables, Guitar leads from kenable, St.Helens, WA11 8LY for CAT6 or (cheaper and with a greater selection of colours) CAT5e leads.
 

nry

Active Member
Feb 22, 2013
312
61
28
I personally use Xenta cables from ebuyer.

Xenta Category 6 Cables | Ebuyer.com

I have had a few dud ones where one of the connectors has been slightly misshapen. But I either crimped a new connector on or just ordered a spare anyway as there about £1 each :)

Most of my cables in my setup are cut from 30m cables I bought from ebuyer and just crimped myself, get the impression this is generally frowned upon in the community but heh haven't had any issues yet.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
I didn't like the idea of re-wiring things with my fumble fingers so got the bits from Quiet PC. I ended up getting both the 5v and the 7v pre-made cables as they were so cheap. Thos are some "big arse" resistors on those leads. How warm will they get? And are they OK that near to the plastic coating of the other wires? I'm not an electronics expert so please excuse the "noob" type of question.



Also got myself a silicon gasket for the 60mm fan to hopefully stop the vibrations. While I was there I got a 120mm to go on the fan in the back of my Antec 300 case too. Not sure if it is that causing the vibrations, but I will check it out and see.

I have ordered the Xenta cables from Ebuyer and they should be here soon as well. Nice 1m cables so I can start tidying up a bit. I hate messy switches with extra long cables hanging out everywhere.
 

Mike

Member
May 29, 2012
482
16
18
EU
Maybe a tiny detail, but i want it out there; Can your motherboard, dont know which one it is, provide that amount of power over the fan headers? I always ran big blowers and chit on supermicro boards, but don't know if just any board will live with these.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Hi Mike

It's a 60mm Fractal Design Fan that I am connecting to the CPU fan socket of an Asus AT5NM10-I motherboard. The fan was running before at 3,000rpm. I have got the little leads with the resistors on them to slow it down a bit as it is very noisy, and a bit overkill for the Atom D510 processor that is in the box.

I hope this makes sense
 

Mike

Member
May 29, 2012
482
16
18
EU
Ok, no worries then. I thought you were getting a server fan in there, usually spinning faster than a jet engine.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Well I have been a little bit busy over in the Hyper-V section of the forum trying to sort out some weird network slowdowns. In the end we pinned it down to two group policies that get applied to domain controller and do some digital signing of SMB traffic. Many thanks to the forum members here for helping me out and replicating the issues on their setups.

I have been doing some tests again with the little Atom D510 system. Added the 7v resistor lead for the 60mm fan and the silicon fan gromit. I thought I would try pulling air into the case and also expelling air from the case.

Pulling air into the case, blowing directly at the heatsink, leaves the temperature at 30C and system temperature at 34C. (Silent running)
Sucking air out of the case, fan right by the heatsink, leaves the temperature at 34C and system temperature at 37C. (A bit noisy)

So pulling air into the case, and silent, it is then :)

Now with the issue of the main server now having the following roles I need to make a few changes:
- Domain Controller
- DHCP Server
- DNS Server
- Hyper-V Host
- File server
- iSCSI storage

This will not be the main Hyper-V server at all, just a few little VMs to keep the network ticking over. Due to the issue with the group policies and digital signing I want to virtualize the domain controller part with the DHCP and DNS into a VM on this box. That is the plan anyway. I have read this is much more safe and secure with Server 2012 R2 than it has ever been before. Any suggestions on this are welcome. By the way the little Atom D510 is going to be used as a second physical domain controller from time to time. I will use it for messing about and testing replication Microsoft Sites stuff etc. Hopefully this will give me a system I can use to recover from, so long as I turn it on once a week or so. I want to try out things like DHCP failover between sites.

Now with that in mind it should then just leave the following roles on the main physical server here:
- Hyper-V Host (Joined to Domain)
- Fileserver
- iSCSI storage

There will later be more dedicated Hyper-V host(s), hopefully in a cluster, connected to this setup that will access the iSCSI storage. So please bear that in mind with the next part. There are two schools of thought now with the introduction of NIC teaming in Server 2012. I have done a couple of little network designs here to show that and make it a bit easier.

First, have a physical NIC, or team for each individual thing. So say: 1 x NIC for management, 2 x NIC for Migration, 3 x NIC for iSCSI and use MPIO.



Second, team all physical NICs together onto the vSwitch and then create vNICs for everything from there. And use some QoS on the vSwitch



Now some of this is all a bit new to me. So iSCSI going direct in the first layout I could implement MPIO. In the second obviously I couldn't as I only have the one vNIC which is supposed to be 10Gbe. Would I need MPIO then? If so then I guess I just add another vNIC and go from there?

I do like the idea of all physical NICs in one team into the vSwitch. The theory is great as you could lose all physical NICs apart from one and all services would still work. Much slower, but they would all still work. This is really what my project here is all about, redundancy and keeping things running in disasters. Plus renewing my lab as well :)

So thoughts, ideas, experiences please ladies and gentlemen?
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Bit of an update. I went for the first option listed above. 3 separate NICs for iSCSI and a team for everything else. One weird thing is that the iSCSI NICs keep insisting that they are on the domain and not just a Private unknown network. I think it may be something to do with the main OS also being the domain controller. I have disabled everything on the iSCSI NICs apart from IPv4. And made sure there is no DNS or default gateway on there too.

Going to set up a VM to hold the domain controller, DHCP, and DNS. Going to set up a physical one too just in case.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Thanks Patrick. Next stage is to build up the Virtual Server Hosts that will run off of this. I already have 3 x Intel DQ77MK motherboards and 3 x Intel Pro 1000 PT Quad Port PCIe cards for them. Important to remember these cards are full height cards, not just half height cards with full height brackets. So cases must take full height cards I am trying to find now the smallest cases that I can. Ideally I want it to all pile on top of each other and be as silent as possible too. This is the small stack so far :)



The top of the RM420 Pro case is 66cm long x 43cm wide. I want to have 3 x Virtual Host servers if I can. So either 3 desktop type ones stacked on top of each other and the switch on top. Or 3 x small Micro ATX tower cases stacked next to each other with the switch on top.

I will be using the i7-3770T CPU and hopefully in passive mode with the same cooler as is in the SAN server. CPU is only 45W but still has some grunt to it.

Intel CPU Core i7 3770T Quad Core IvyBridge Processor OEM - CM8063701212200 - Scan.co.uk

I have seen a variety of cases so far and can't make up my mind. I want to use the fanless PSUs again as they seem great. Just need to make sure there is a bit of airflow in the case. For them and the passive CPU cooler. Not sure how the fanless PSU would cope sat at the top of a tower case. Is anybody else doing this?

Here are some of the cases I am looking at.

Ace Ecco 250 Black Micro-ATX Case with Sound Proofing w/o PSU - ECCO 250 - Scan.co.uk
Silverstone Precision PS07B USB 3.0 Mini Tower Micro-ATX Case No PSU - SST-PS07B USB 3.0 - Scan.co.uk
Silverstone Temjin Evolution Black Mini Tower Case with 180mm Quiet Fan no PSU (ATX/PS2) - SST-TJ08B-E - Scan.co.uk
Silverstone Precision PS08B micro-ATX Mini Tower Case with USB 3.0 no PSU (Standard ATX PSU Support) - SST-PS08B USB 3.0 - Scan.co.uk
Silverstone PS09B USB 3.0 Precision microATX Case with Foam Padding for Silence - SST-PS09B USB 3.0 - Scan.co.uk
Fractal Design Core 1000 micro-ATX Case takes Standard ATX PSU - FD-CA-CORE-1000-BL - Scan.co.uk
Coolermaster Silencio 352 Matt Black Micro-ATX Mini Tower Silent Case - SIL-352M-KKN1 - Scan.co.uk
Nanoxia Deep Silence 4 Black Low Noise mATX PC Case - NA-DS4-BLACK - Scan.co.uk
Silverstone GD05B USB 3.0 Grandia HTPC Black Micro ATX Case - SST-GD05B USB 3.0 - Scan.co.uk

There may well be other cases and I am open to suggestions please.

Has anyone else managed to get the Intel DQ77MK motherboard to boot from iSCSI off of one of the integral NICs? I'd like to do this and have the hypervisor OSes stored on the SAN so I can easily switch between Hyper-V and ESXi
 
Last edited:

nry

Active Member
Feb 22, 2013
312
61
28
Looking good :)

With regards to booting from iSCSI, while I can't answer for your particular motherboard, I have had nothing but success with iPXE.
I have pfSense setup as a network boot server with an iPXE image, basically a simple boot loader which gets downloaded to any client that requests it.
This then makes a HTTP request to a local web server with it's MAC address (eg http://pxe.example.com/00:00:00:00:00:00)

I can then setup individual config's for each MAC address which define an iSCSI target to mount and boot from. As I use PHP for this, I have a very basic control panel which lets me switch between configs, eg boot server 1 into ESXi, Ubuntu Server or Windows 7 (haven't got this one working yet fails to install)
If no config is setup, I have a boot menu loaded which lets me pick from installer disks etc

Far from perfect but I have had this working on old AM2 motherboards no issue. The alternative is to install iPXE to a USB stick and boot from that.

PS can't help but think you need one of these DELL PowerEdge C6100 4 Node 8x Xeon Quad Core 2.26GHz 32GB DDR3 Cloud Server VTx | eBay ;)
 

OBasel

Active Member
Dec 28, 2010
494
62
28
I'm still amazed that nobody has come up with a really good netboot framework. Think of all the DCs that run diskless compute nodes nowadays and all of the cloud instances. bootstrap minimal AMI and you're off. closest thing to that but I don't really want to setup Openstack just for that.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Looking good :)

With regards to booting from iSCSI, while I can't answer for your particular motherboard, I have had nothing but success with iPXE.
I have pfSense setup as a network boot server with an iPXE image, basically a simple boot loader which gets downloaded to any client that requests it.
This then makes a HTTP request to a local web server with it's MAC address (eg http://pxe.example.com/00:00:00:00:00:00)

I can then setup individual config's for each MAC address which define an iSCSI target to mount and boot from. As I use PHP for this, I have a very basic control panel which lets me switch between configs, eg boot server 1 into ESXi, Ubuntu Server or Windows 7 (haven't got this one working yet fails to install)
If no config is setup, I have a boot menu loaded which lets me pick from installer disks etc

Far from perfect but I have had this working on old AM2 motherboards no issue. The alternative is to install iPXE to a USB stick and boot from that.

PS can't help but think you need one of these DELL PowerEdge C6100 4 Node 8x Xeon Quad Core 2.26GHz 32GB DDR3 Cloud Server VTx | eBay ;)
Thanks, still not a patch on your set up yet :)

iPXE looks really good. I already have a virtual server setup to run Microsoft WDS and MDT that uses PXE boot. Will this cause a problem? The DHCP server provides the IP address for that server. Changing the boot config via a HTTP server would be much easier than keep fiddling with BIOSes or NIC firmware. I wonder if I could use a different VLAN for the NIC loading the the hypervisor via iSCSI? That way I can change the DHCP scope options for the PXE server. Hmmmm that might just work

I was going to re-flash the Intel NIC ROM to use the iSCSI firmware instead of the PXE firmware. Or was hoping the BIOS of the board may allow iSCSI boot by default. Need to get the rest of my hardware for the virtual host servers first really and do some testing.

Biggest decision is the case to get for them.

I'd love a C6100 but they are so big and the power consumptions look massive. I am hoping to keep the whole final stack around the 500W-600W mark. So that is SAN, switch, router, 3 x virtual host servers.
 

nry

Active Member
Feb 22, 2013
312
61
28
Not too sure about WDS as I haven't used that, but a quick Google returned this thread over at the iPXE forums, [SOLVED] : iPXE and WDS Config

Looks like someone else had the same idea, only scanned this thread but if you went down this route it looks like you would want to always boot iPXE, then you could have a script which simply loads and boots the WDS software. That way everything stays on a single VLAN.

The C6100 is tempting but my main concern is the noise, really wish I had somewhere isolated to keep all my kit!
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Not too sure about WDS as I haven't used that, but a quick Google returned this thread over at the iPXE forums, [SOLVED] : iPXE and WDS Config

Looks like someone else had the same idea, only scanned this thread but if you went down this route it looks like you would want to always boot iPXE, then you could have a script which simply loads and boots the WDS software. That way everything stays on a single VLAN.

The C6100 is tempting but my main concern is the noise, really wish I had somewhere isolated to keep all my kit!
Ahhhhhhh thanks :) That looks similar to what I was thinking. Just need to solve my case dilemma first. I know what you mean about noise from kit. That is why the cases for the Virtual Host servers are such a big decision for me.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
OK, so narrowed down the choices of case for the virtual host server to two now.

1) Silverstone GD05B USB 3.0 Grandia HTPC Black Micro ATX Case - desktop type case




I was thinking about running it back to front with the rear of the case facing the same way as the ports on the switch. Makes for much easier cabling. They can stack on top of each other with the switch on the very top.

I also saw this Highspeed PC - ATX Power Bracket / Switch at Xoxide! and thought it would be nice to have something like it to go in the one of the rear empty expansion slots. This one is a bit pricey though and not in the UK




I have seen a few complaints about people getting static shocks from the front of this case. If I use the parts I want though, including the fanless SeaSonic power supply then the only moving parts will be the three case fans. I should be able to control their speed from the BIOS of the Intel DQ77MK motherboard with any luck.

2) Silverstone Temjin Evolution Black Mini Tower Case





Nice small case and will fit 3 across the top of the RM-420 Pro chassis. Should also leave a nice small gap in between to run the network leads from the back to the switch that will sit on top. My only concern is if the fanless power supply would work in there. Looks awfully tight to me. There seems to be room above and below the power supply but none at the sides, and an air vent on the top of the case too. Not sure I'd like to risk it with no direct fan blowing at the power supply. If not then I would be looking for a silent fan gold/platinum rated 300W or 400W power supply. Any recommendations please?
 

nry

Active Member
Feb 22, 2013
312
61
28
Going to throw it into the mix, have you considered either of these from Xcase?
2u 6 bay rackmount server case
X-Case RM 204 short 2u rackmount server pc case

I have one of these PSUs too (sorry you may have said you already purchased fanless PSUs?)
Antec.com - Product: EA-380D Green
Very quiet, pretty sure I paid about £30-40 for it.

If not I'd get one of the cheaper mATX cases.
Fractal Design Core 1000 micro-ATX Case takes Standard ATX PSU - FD-CA-CORE-1000-BL - Scan.co.uk

Looks nice and simple.

EDIT: just read you have full height NICs, bit of a pain that one!
I do have 2 low profile 10GbE NICs to sell and one full profile one ;)
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Going to throw it into the mix, have you considered either of these from Xcase?
2u 6 bay rackmount server case
X-Case RM 204 short 2u rackmount server pc case

I have one of these PSUs too (sorry you may have said you already purchased fanless PSUs?)
Antec.com - Product: EA-380D Green
Very quiet, pretty sure I paid about £30-40 for it.

If not I'd get one of the cheaper mATX cases.
Fractal Design Core 1000 micro-ATX Case takes Standard ATX PSU - FD-CA-CORE-1000-BL - Scan.co.uk

Looks nice and simple.

EDIT: just read you have full height NICs, bit of a pain that one!
I do have 2 low profile 10GbE NICs to sell and one full profile one ;)
Yes sadly full height cards :( Are those single 10Gbe NICs? What model? That means I'd need a 10Gbe switch then too. Damn it!!!!

I don't have the power supplies yet. I did get one of the SeaSonic fanless ones for the main server though. 520W model. It is brilliant. With fanless you just knows that it will be silent. You do need a decent element of airflow within the case though. If only the Temjin or the Core 1000 had the power supply mounted at the bottom of the case instead of the top.
 

ATrain

New Member
Feb 11, 2014
4
0
1
Yes sadly full height cards :( Are those single 10Gbe NICs? What model? That means I'd need a 10Gbe switch then too. Damn it!!!!

I don't have the power supplies yet. I did get one of the SeaSonic fanless ones for the main server though. 520W model. It is brilliant. With fanless you just knows that it will be silent. You do need a decent element of airflow within the case though. If only the Temjin or the Core 1000 had the power supply mounted at the bottom of the case instead of the top.
Graham, I have a fanless Seasonic PSU in the Temjin case. It works great. The only issue is that there is not much clearance between the PSU and an optical drive in the top slot. Leave the top vent uncovered and you will have no problems.
 

TallGraham

Member
Apr 28, 2013
143
23
18
Hastings, England
Graham, I have a fanless Seasonic PSU in the Temjin case. It works great. The only issue is that there is not much clearance between the PSU and an optical drive in the top slot. Leave the top vent uncovered and you will have no problems.
Ah ha! That is brilliant ATrain. Thanks so much for the feedback :)

Nothing like a response from someone who actually has the hardware. I won't be running an optical drive at all so no worries there. I was looking at the 400W Fan less model. I don't think they make a smaller one. What do you have in yours if you don't mind me asking?

Does much heat come out the top vent? Do you run a rear fan as well? I have heard the front fan can get a bit noisy unless you slow it down a bit. I would really appreciate your input and thoughts?