Hyper-V Web Hosting Cluster Exploration

MiniKnight

Well-Known Member
Mar 30, 2012
2,997
907
113
NYC
I recommend that you move off of Proxmox, and move into something less GUI-dependent (Debian, CentOS, etc.)
From reading his previous posts, I think he likes the GUI.

I posted this in your other thread. Change the 2x pfsense boxes to Rangeley with ESXi 5.5 and pfsense in VMs. That gives you the 4x NICs you need and have in your current setup.

Have 2x 1U short depth chassis for the ESXi/ pfsense. Use USB storage for them. Make 2x USB so if one drive dies you can boot off the other. Make little VMs in them with big 3TB drives which you can mirror and use as backup.

Behind those have your Hyper-V cluster. Use a single C6100. If the C6100 goes down, you could probably run off of one of the ESXi/ pfsense boxes for awhile. Wordpress/ vbulletin have tons of cached pages so not too bad on processors for sites like this.

If you do that, you are all set at a very low cost. You could probably sell one running and one spare C6100's that are in there too. You'll end up with 2U full free. 2U that you could probably put short-depth chassis into one side. I'd also venture to guess that filling a C6100 with 4 instead of 3 active nodes would be more efficient and you could offload the other C6100.

I'd bet in the process, you would have more free space PLUS (and here is the big one) you would save 1.5-2A of your 5A limit. You would still get your GUI. pfsense on ESXi and P2V for that is absolutely trivial.

Only thing you might have if Proxmox is sucking is the potential that you get a few minutes of downtime while you move the 2x C6100 nodes to the other chassis. Since you do this enough, <10 minutes of downtime.
 

Attilas

New Member
Jan 9, 2014
3
0
0
I have a 2 hosts + shared storage at home since 2 years for fun, in a big 42U rack mount that my girlfriend don't like !! Been using ESXI mostly but I recently (2 months) changed to Proxmox. I tried 2 times Hyper-V at home with access to a Windows 2012 Standard Licence. Two things that sounded wrong for me:

Mostly I found Hyper-V to be way too tight with the Microsoft Active Directory, File share access and security settings. For migration to work, you have to setup a bunch of access, delegations and cluster storage services. Im not a Microsoft expert, but i went way to much on the internet to find out cures for error code. In the end, I felt that the Hyper-V Hypervisor is not independent of the Virtual Guess OS it runs and/or Microsoft AD. Sounded to me it was not an "real abstract layer of virtualization" compared to ESXI and Proxmox which both live in their own world.

Other things I really disliked from Hyper-V is the accessibility of management tools. Since it is not a downloadable setup file and need to install it from the "Add/remove Windows features" screen, you are stuck using Windows 8 OR Windows 2012 to perfectly manage the cluster. If you have Windows 7, there is a management tool available but it is a prior version that don't have all functions of the 8 or 2012 ones. Ending mostly RDP to have access to management tools. In the end I found it much worst that the vSphere management, that at least can be installed on any Windows version that support .NET and now mostly any OS with ESXI vSphere WebAccess.

Other than those two major set back for me. I found it stable, really fast, accessible and polished. But, again you need to really hug microsoft ecosystem and versioning there.
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,489
228
63
That was really insightful Attilas.

Can you use any Windows 8 machine to manage? Even a Bay trail tablet?
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
Heading to the colo en route to the airport to install a few more SSDs. Going to think about this a bit while there and use the opportunity to triple check that I have everything documented properly. I also think one of the pfSense nodes stopped responding so want to check that out.

Great points on Hyper-V though and certainly something I am evaluating.
 

PigLover

Moderator
Jan 26, 2011
2,967
1,280
113
Note that in 2012 R2 MS removed most of the ties between migration and their clustered file system. Migration works well with little more than AD and a shared SMB filesystem.

Otherwise, Atlas' comments are correct. It is very tightly tied with AD and you do have to have Windows 8 to effectively manage it.
 

Attilas

New Member
Jan 9, 2014
3
0
0
That was really insightful Attilas.

Can you use any Windows 8 machine to manage? Even a Bay trail tablet?
Thanks.

If you are running Windows 8 Pro (which include Hyper-V installable service btw) you can install the management tools in "add/remove windows features". So im confident any Atom tablets with a full Windows 8 Pro will have access to management tools. Just checked Windows 8.1 on Surface 2 and I don't have management tools. Surface Pro most likely have it.

But to add to my point. Let just look at my job. We all have Windows 7 Pro workstations. That mean I can't even completely manage a Windows Server 2012 Cluster from my workstation. And now that Windows Server 2012 R2 is out, I guess they upgraded the Management tools at the same time and you may need an upgrade to Windows 8.1 to gain access to the matched management tools. I didn't test this out, since my last try was with Windows 8 and Windows 2012 Standard R1. The management tool should be a downloadable package that can be freely matched to your cluster in my humble option.

Note that in 2012 R2 MS removed most of the ties between migration and their clustered file system. Migration works well with little more than AD and a shared SMB filesystem.

Otherwise, Atlas' comments are correct. It is very tightly tied with AD and you do have to have Windows 8 to effectively manage it.
That is interesting. So we don't need their "clustered file service" anymore ? Does a NFS share (which support file level lock) would work now ?
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
So I have been playing around a bit with this. There is still the issue of the fact that when proxmox1 (running nothing) goes down, the virtual machine (using local storage) on proxmox4 running the forums becomes unreachable.

Unsure of why this is.

Interestingly enough, I was able to:
1. copy the http://servethe.biz VM over to the Windows 8 VM.
2. rename the .raw disk to .img
3. use Starwind V2V to convert the disk to .vhd
4. made a VM in Hyper-V with the .vhd
5. booted the VM
6. turned the proxmox VM off
7. changed the NAT mapping to map the external IP to the Hyper-V VM

Here is the really interesting bit. I used pingdom's page speed tester before and after the swap. Load times ranged as follows:
Proxmox VE: 3.59s - 4.24s
Hyper-V: 2.36s - 2.68s

I am not sure I would be ready to say it is faster yet, but it certainly does not appear to be slower.
 

Attilas

New Member
Jan 9, 2014
3
0
0
I also felt Hyper-V to be snappier than ESXI last time I tried. I would say Proxmox feel in the range of ESXI imho.

That is some really interesting performance increase indeed.

I always tough Microsoft should be the best placed company to virtualize their own OS better than any other hypervisor.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
The extremely interesting thing is that the VM in question was running in a Ubuntu 13.x VM on Hyper-V.

Just as an update on this: tonight I am going down the path of somewhat no return. Replacing Proxmox VE on a second node. Officially half of the datacenter will be on Hyper-V and half on Proxmox after tonight.
 

PigLover

Moderator
Jan 26, 2011
2,967
1,280
113
With the Hyper-V "Linux Integration Services" getting mainlined into the Linux kernel its getting easier and easier to use Hyper-V. CentOS & RHeL >= 6.4, SUSE >= 10.4, Fedora >= 18, Ubuntu 12.4, 12.10, 13.4 & 13.10 and many others all have Hyper-V LIS pre-compiled in the standard release.

The advantages of the Hyper-V vSwitch over the simple Linux Bridge-mode networking used by Proxmox VE could explain the improvements you see. Remember that Proxmox VE's bridging disables almost all hardware accelleration in the NIC card. No TCP checksum offload, no large send offload, no receive side scaling, nothing. With the Hyper-V vSwitch all of those features are not only in play - but on better ProSet NICs (like the ones in your C6100) some specialized visualization enhancements that you don't normally use are enabled too.

Note that this isn't an indictment of KVM networking in general - just the choices made by Proxmox VE team not to include a proper vSwitch and to just rely on Linux bridge mode operation. KVM plus openvswitch, for example, has network performance that is excellent. But openvswitch is incompatible with OpenVZ containers so it can't be used in Proxmox VE.
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
Great information. I know the Dell C6100 hardware with short CAT6 cables and the HP V1910-24G switches are capable of decent transfer speeds but I have been seeing 4-5MB/s transferring some images between Proxmox and Hyper-V. Strange because these are sequential transfers SSD to SSD.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
21 January 2014 - Status update
The current configuration of the four nodes is the following:
Nodes 1 and 2 are now converted to Hyper-V Server 2012 R2 base - so far only one domain controller
Nodes 3 and 4 are still configured as Proxmox VE

I currently have http://servethe.biz, a Windows 8.1 VM and a Windows Server 2012 R2 all sitting on Node 2's main Intel 160GB SSD. Node 1 has a 3TB WD Red and a 120GB SSD (sandforce that will likely become used for ISOs) and Node 2 has a Kingston E100 400GB SSD.

The current tasks are:
1. Figure out how to configure the storage
2. Explore using CentOS or Ubuntu with Samba 4.0 as the domain controllers/ DNS - PigLover is right and realistically <5-10 logins are needed
3. Test with ST.B
4. Migrate the main site and forums if all goes well

Fun learning this part.
 

PigLover

Moderator
Jan 26, 2011
2,967
1,280
113
A note on the Samba4 Domain Controller tutorial I posted:

- As written if you follow it verbatim it will clone the development nightly from Git. This turns out to be a development build 4.2.x. For something like STH you probably want to stick to the current stable release which as of today is 4.1.4.

- The instructions are almost perfect. Using a minimal install of Centos 6.5 server (no gui) every step worked perfect with one exception: you need to install one additional pre-requisite package 'openldap-libs' (this may be related to the 4.2.x development build and may not be needed for 4.1.4).

- When you get to the step where you run samba-tool to provision the domain add the flag '--use-rfc2307'. This will include the Unix domain extensions needed to maintain uid/gid information for unix/linux/solaris users.

So far I built the domain controller but haven't tried to join a Windows workstations to it or manage it. On airplanes this week and tied up over the weekend so it might be a while before I get to it.
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
I tried making a failover cluster. I think I am running into the issue where you need a unique SID... but even if I fixed that it seems like there is a long way to go to make that work. The book I read made it sound so easy :)
 

Marsh

Moderator
May 12, 2013
2,292
1,107
113
Sysprep the VM would fixed unique SID before joining domain.

I used powershell scripts to build failover cluster for my lab, if you like, I could zip it up and email you.
 

TangoWhiskey9

Active Member
Jun 28, 2013
402
59
28
That costs more than a Windows 8.1 VM. Almost as much as 2012 R2 Standard or essentials to just manage everything.

Read the site, I really don't see what you get over Sever Manager + Hyper-V manager
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,926
113
27 January 2014 update - a bit slower progress due to the fact that I have been on planes constantly for the past week.

Went to Tools -> Failover Cluster Manager and made a failover cluster today. Did the no Microsoft support option and that helped quite a bit.

The issue right now is that it will not accept any of the loaded disks. I read that failover cluster manager requires SAS and all three/ four disks are set to SATA AHCI. I was thinking about using this registry hack but not sure I want to go that path yet.

I tried doing Storage Spaces on the the Hypervisor using both a 3TB WD Red and a 120GB SSD. The WD RED was reported as media type unknown so I used:
Code:
set-physicaldisk -friendlyname "PhysicalDisk1" -mediatype HDD
(The SSD was PhysicalDisk0)

That allowed me to make the storage space using automatic tiering. I must say, I am somewhat bummed that the cluster storage bit is so difficult.