Hyper-V Server on Avoton

[PigLover]

Here's a small server I just put together to run a few VMs.

Goals:
- Hyper-V host to run the things that need to be "always on". A small server so that I can turn everything else in the lab off.
- Host all required disk storage locally (AIO) so that it remains functional if file server hosts are down
--- Note: disk resiliency is not a high priority as files will be backed up regularly (constantly using a change monitoring backup program)
- Fit into existing rack (1U)
- As low power as possible
- As quiet as possible

- Will run (at least) the following VMs:
--- FreePBX for phones & voice mail
--- Windows 7 host to manage security cameras (Blue Iris - currently w/3 1080p cameras)
--- Windows 7 host with PlayON (proxy's media sites like Hulu & Netflix for multiple non-windows media players)
--- DVR - MythTV backend running on Ubuntu
--- Web server used for simple remote upload & access for photos & video. Currently IIS on Windows Server 2012 but re-building on CentOS/Apache to avoid licensing issues
--- Samba4 based Active Directory on CentOS

I managed to get a surprisingly good deal on a brand new SuperMicro 1U with 4 3.5" hotswaps and a 350w Gold PSU (813MTQ-350CB). It was actually a SuperServer barebones with a mATX socket 1150 MB (5017C-MTF) and I landed the whole thing for $172.50 shipped on eBay. So I pulled out the the IvyBridge MB and will re-use it elsewhere.

Final configuration:

Chassis/PSU: SuperMicro 813MTQ-350CB 1U
Motherboard: SuperMicro A1SAi-2750F mini-ITX Avoton. 8 Core, 2.4Ghz
Memory: 4x Kingston KVR16LSE11/8 8GB 1.35v ECC Unbuffered SO-DIMM
Boot drive: 120GB Mushkin mSATA in PCIe x1 carrier for power only
SSD: Sandisk Exteme 489GB
Hard Drives: 4x WD Red 4TB

Given the chassis it probably would have been better to get the mATX Avoton board, but I bought the miniITX previously without a clear plan around it.

The mSATA boot drive and carrier were leftovers from an earlier project. No real good reason not to have just used a regular SSD. Also for the SanDisk Extreme 480 - a leftover.

I'll post some pics tomorrow and then give a rundown on setting it up and results. Just a summary for now: all VMs fat dumb and happy. Was worried about whether it could handle the video cameras but it seems to have just enough. With all four drives spinning, all three video cameras recording and two programs recording in HD on the DVR it sits just under 50watts (prior host - DL180G6 - sat well over 200w for the same load).

 

[Patrick]

Nice! That is a big enough power savings figure to make a noticeable power difference.

Let me know if you want to host pictures on the main site.

 

[PigLover]

Yeah - it is a good savings. It all started before we left for a bit around Christmas time and I wanted to shut down the lab. But I kept discovering dependencies. Shut down the file servers...nope - need them to store the video from the cameras and DVR. Can't shut down the Active Directory host (even though nothing "necessary" was on AD) because it also hosted the DNS. So the "shutdown" version still had 600+ watts worth of equipment running. 15kwh/day. I can certainly afford the $3/day (LOL) but it felt irresponsible.

So i decided to see if I could build a single "must have" machine at low power (and noise).

I do have plans to add a couple more cameras and am a bit concerned that I could be too near the limits of what this can support. But I think this is getting quite a lot out of the little Avoton.

Oh - I also moved the "current" part of the DVD library onto this server too so that my petsitter/housesitters will be able to watch movies when we are away. About 4TB moved. The four-link GigE held nice solid 400MB/s transfers using SMB3 multipath. Very pleased.

 

[Patrick]

Oh - I also moved the "current" part of the DVD library onto this server too so that my petsitter/housesitters will be able to watch movies when we are away. About 4TB moved. The four-link GigE held nice solid 400MB/s transfers using SMB3 multipath. Very pleased.

I am a bit jealous to say the least!

The DC has one Hyper-V host up but until I can get that second Hyper-V host up (or more specifically the first Proxmox host down and install Hyper-V on it.)

 

[PigLover]

As promised. A few pics.

Your basic 1U Supermicro 813MTQ-350CB:

Inside with the A1SAi-2750F, mSATA and Sandisk Extreme 480. All fits super easy - except that the PCIe riser doesn't line up with the miniITX motherboard. Its designed for microATX so a short flex riser to get it all to fit. I'm just using the PCIe for power anyway. Also note the orange SATA cable. The stock cables in the case were too short to reach the sata plugs on the miniITX. I had to take out the shortest one and shift the other three over to make things reach - then the only cable I had around that was long enough and had a right-angle end for the backplane happened to be ugly-orange.

Little closeup on the mSATA adapter. Its just cute...

Lastly - with everything running. 44 watts is typical with Hyper-V with 5 VMs, one of them taking continuous feeds from three 1080p IP cameras. When the cameras see motion and start recording and MythTV kicks off a recording from TV I've seen it tick up to 51w. Not too bad for 8 cores, 32GB, 4x 4TB spinning disks and a couple of SSDs.

A quick note about noise: only three of the four fans are actually connected because the MB only had three fan headers. As soon as the POST completes all the fans spin down to nice quiet operation. Its a far cry from silent but I was really worried about how the little 40mm fans in the chassis and PSU would sound. Turns out it I'm very satisfied.

 

[mackle]

Nice!

I like the approach and want to do something similar myself - thanks for sharing the build out.

 

[PigLover]

A few notes on setting it all up:

Installing Hyper-V Server 2012 R2:
Very, very simple. The MB Sata-3 ports are recognized out of the box. Other than picking which disk to install to there are pretty much no options. On reboot the only question you get asks you to set the Administrator password.

After it stars up with Hyper-V Server its missing device drivers. It starts up to a screen that looks like this:

Loading device drivers:
If you select the option for "network settings" you'll see there are no NICs recognized. You've got to load the drivers. Luckily, and unlike earlier versions of Hyper-V Server, this is now pretty easy. Standard driver install files (autorun's or setups) run from the console. Just download the drivers from SuperMicro's web site (or your MB makers site for other boards), load them up onto a USB stick and then run the installers from the Command tool on the Hyper-V Server console. As usual loading drivers on Windows - load the chipset drivers first, then the LAN, then others - though there is probably no need to load the video drivers here.

Setting up to manage from a workdroup computer (management without a Domain Controller):
This part ought to be easy...but MS still makes it a PITA. Though to be fair it is much, much easier than it used to be. Using Server Configuration menus do the followig (1) give your computer a name [Menu 2] - you'll have to reboot after this one (2) enable remote management [Menu 4] (3) Enable ping response for testing [Menu 4 option 3] (4) Enable remote desktop for all clients [Menu 7]. Then configure your NICs. I strongly recommend that you set up at least one NIC on a static address. Then you should probably get current on security updates [Menu 5 to enable automatic updates and Menu 6 to do an update right now].

When you've done all this you'd think you are done. But you're not...by default the Windows Firewall running on your net Hyper-V server will recognize your LAN as a "public" network. And the default rules for a "public" network doesn't allow connections for Remote Desktop (RDP) or remote management (RSAT or Hyper-V Manager). The easiest way to deal with this is just disable the windows firewall on the Hyper-V Server (you can turn it back on later once we have management access).

Use the following command on the console window on the new Hyper-V Server:

netsh advfirewall set allprofiles state off​

Now - back to your Windows machine in the next post.

 

[TangoWhiskey9]

Great looking pictures/ thread. I had been using this one: http://forums.servethehome.com/wind...-v-manager-without-domain-only-workgroup.html

 

[NeverDie]

Here's a small server I just put together to run a few VMs.

Goals:
- Hyper-V host to run the things that need to be "always on". A small server so that I can turn everything else in the lab off.
- Host all required disk storage locally (AIO) so that it remains functional if file server hosts are down
--- Note: disk resiliency is not a high priority as files will be backed up regularly (constantly using a change monitoring backup program)
- Fit into existing rack (1U)
- As low power as possible
- As quiet as possible

- Will run (at least) the following VMs:
--- FreePBX for phones & voice mail
--- Windows 7 host to manage security cameras (Blue Iris - currently w/3 1080p cameras)
--- Windows 7 host with PlayON (proxy's media sites like Hulu & Netflix for multiple non-windows media players)
--- DVR - MythTV backend running on Ubuntu
--- Web server used for simple remote upload & access for photos & video. Currently IIS on Windows Server 2012 but re-building on CentOS/Apache to avoid licensing issues
--- Samba4 based Active Directory on CentOS

I managed to get a surprisingly good deal on a brand new SuperMicro 1U with 4 3.5" hotswaps and a 350w Gold PSU (813MTQ-350CB). It was actually a SuperServer barebones with a mATX socket 1150 MB (5017C-MTF) and I landed the whole thing for $172.50 shipped on eBay. So I pulled out the the IvyBridge MB and will re-use it elsewhere.

Final configuration:

Chassis/PSU: SuperMicro 813MTQ-350CB 1U
Motherboard: SuperMicro A1SAi-2750F mini-ITX Avoton. 8 Core, 2.4Ghz
Memory: 4x Kingston KVR16LSE11/8 8GB 1.35v ECC Unbuffered SO-DIMM
Boot drive: 120GB Mushkin mSATA in PCIe x1 carrier for power only
SSD: Sandisk Exteme 489GB
Hard Drives: 4x WD Red 4TB

Given the chassis it probably would have been better to get the mATX Avoton board, but I bought the miniITX previously without a clear plan around it.

The mSATA boot drive and carrier were leftovers from an earlier project. No real good reason not to have just used a regular SSD. Also for the SanDisk Extreme 480 - a leftover.

I'll post some pics tomorrow and then give a rundown on setting it up and results. Just a summary for now: all VMs fat dumb and happy. Was worried about whether it could handle the video cameras but it seems to have just enough. With all four drives spinning, all three video cameras recording and two programs recording in HD on the DVR it sits just under 50watts (prior host - DL180G6 - sat well over 200w for the same load).

I'm curious as to whether the SuperMicro A1SAi-2750F turned out to be more than adequate, or inadequate, or just right for handling the loads you listed?

 

[PigLover]

I'm curious as to whether the SuperMicro A1SAi-2750F turned out to be more than adequate, or inadequate, or just right for handling the loads you listed?

Pretty much perfect. Its lightly loaded enough that I could still add some more. But I'm also adding more cameras to Blue Iris soon and that could take up the headroom I have left.

I'm very satisfied with it. My only complaint is that the 4x 4TB spinning disks draw almost as much power as the rest of the system combined

 

[NeverDie]

I'm very satisfied with it. My only complaint is that the 4x 4TB spinning disks draw almost as much power as the rest of the system combined

Pretty soon I'll going to be facing the same issue. I guess maybe some kind of hierarchical storage? I don't see it discussed much.

 

[PigLover]

In a sense my storage is already "hierarchical" (sorta - poor mans style). The VMs themselves all live on the 480Gb SSD and are replicated daily using Hyper-V replication. The spinners are configured as a Storage Spaces "simple" pool (aka Raid-0). Its pretty fast, but generally considered risky to do that much Raid-0 with large drives. In this case none of the data is considered really high value. The DVD library is a subset of a larger library and is fully replicated on another server. The Cameras collect motion detected clips and are replicated to cloud storage immediately upon close. And the DVR files are replicated daily to a backup server (as long as the backup server is on).

I actually have a brother for this one all built. Difference is that it is the SuperMicro mATX Avoton board inside a 113MTQ chassis (8x 2.5" drives in 1U). It has a 9207-8i to get to the drives. Right now I'm waiting on the "right" deal for larger SSDs and then I'll see how it does for power. The LSI card probably draws 7-8watts and might offset most of the savings from losing the spinners.

 

[NeverDie]

How about sending the videocam streams to a single drive so that the other drives (holding the DVD collection) can spin down until needed?

 

[PigLover]

How about sending the videocam streams to a single drive so that the other drives (holding the DVD collection) can spin down until needed?

Yup. Logical and makes sense. Current problem is that using Storage Spaces disables spindown on the drives in the pool. I've been toying with various pooling over filesystem approaches to see which one I like best (DriverBender, DrivePool, FlexRaid, etc) but haven't really gotten around to trying them out on the Hyper-V server machines. Will be interesting to see which installers work on the "no GUI" server-core you get with Hyper-V server.

The project list is too long...this one might pop to the top soon but for now I've been more interested in some changes I've been making to downsize/simplify my networking and getting the file servers more to my liking (quieter, lower power, etc). I may post some of it in the coming months.

 

[NeverDie]

Yup. Logical and makes sense. Current problem is that using Storage Spaces disables spindown on the drives in the pool. I've been toying with various pooling over filesystem approaches to see which one I like best (DriverBender, DrivePool, FlexRaid, etc) but haven't really gotten around to trying them out on the Hyper-V server machines. Will be interesting to see which installers work on the "no GUI" server-core you get with Hyper-V server.

The project list is too long...this one might pop to the top soon but for now I've been more interested in some changes I've been making to downsize/simplify my networking and getting the file servers more to my liking (quieter, lower power, etc). I may post some of it in the coming months.

Interesting. Maybe for that reason this is a case where virtualizing onto one ECC server is more hassle than it's worth. Or, looked at another way, virtualizing isn't a hassle, but getting another 12 watts of power savings is more hassle than its worth. I'm unsure which perspective is right.

 

[PigLover]

Its an issue of recognizing the challenge and then overcoming it. Is the next 12 watts worth it? Of course not. But figuring out how to get it and then successfully deploying it definitely is. The same methods might be used later for 1200 watts in another context.

Live for the challenge - and then overcome it.

 

[lboregard]

Thanks for the post PigLover, it was very helpful as I replicated the setup.

Connecting to the HyperV server from a Windows 8.1 Pro workstation, took me approx 4hrs, 4 and a half.

It's seriously complicated, but not surprising coming from Microsoft.

I'd like to add that I wouldn't have been able to do it without HVremote (Configure Hyper-V Remote Management in seconds - John Howard - Senior Program Manager in the Hyper-V team at Microsoft - Site Home - TechNet Blogs

It needed firewalls running on both server and client, but after that it worked.

I'm still unable to connect to the server via remote desktop, so I still need to learn some more.

 

[MiniKnight]

Thanks for the post PigLover, it was very helpful as I replicated the setup.

Connecting to the HyperV server from a Windows 8.1 Pro workstation, took me approx 4hrs, 4 and a half.
....

I'm still unable to connect to the server via remote desktop, so I still need to learn some more.

Way too complex. I have this bookmarked: https://forums.servethehome.com/ind...v-manager-without-domain-only-workgroup.2998/

 

[lboregard]

Time to do some bookmarking