1TB 2.5" HDD for Hypervisor system

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

PigLover

Moderator
Jan 26, 2011
3,186
1,545
113
For a small home network windows as DNS/DHCP only makes sense if you are running an Active Directory domain. Otherwise you should just run it on your router.
 

HellDiverUK

Active Member
Jul 16, 2014
290
52
28
47
To be fair, I don't see any reason to use pfsense in a home situation, either. Why fanny about with a PC and multiple NICs when any decent router will do it all.? If your router's software doesn't do what you need, I'm pretty sure one of the open source router firmwares (TomatoUSB, DD-WRT, etc) that can be flashed on to common consumer routers will do everything you need.

An Asus RT-N66U or similar with TomatoUSB is probably the only router anyone will ever need. You can even do bond bonding for a 2Gb link if you're running Merlin-RT formware on certain Asus routers...
 

spazoid

Member
Apr 26, 2011
92
10
8
Copenhagen, Denmark
To be fair, I don't see any reason to use pfsense in a home situation, either. Why fanny about with a PC and multiple NICs when any decent router will do it all.?
Stability. I got tired of rebooting my router every now and then.
Flexibility. Paying 200 bucks just to upgrade the access point? PFsense + dedicated AP's work much better and costs less.
Scalability. Lots of routers only have 100 mbit WAN NICs. This might not be a problem for most, but it could easily become one. PFsense on proper hardware also supports a lot more connections than even modern routers. Again, not an issue for most, but it might become one as IoT is getting more and more widespread.
Customisation. Lots of fun packages to play around with. Which leads me to my last point:
FUN! I'm sure most of us with multi-TB arrays, hypervisors, Infiniband networks, PoE switches etc. NEEDS it. We WANT it. We tinker with it, we break it, we fix it, we brag about it, we learn from it. Off-the-shelf routers are just appliances. They're boring.
 

Mike

Member
May 29, 2012
482
16
18
EU
Even Pfsense is more up to date security wise than most of the router firmwares available. Also it is used to be cheaper to set up a router distribution than to get the top of the line Atheros rebranded router and still run into the limits with a high # connections and dead slow interfaces.
 
  • Like
Reactions: andrewbedia

HellDiverUK

Active Member
Jul 16, 2014
290
52
28
47
You're obviously buying the wrong routers then...

RT-AC68U with TomatoUSB Shibby, and stick on optware, and all your points are moot. I have a N66U which is pretty old, and I'm running optware on there. All the storage is on a microSD card in the slot inside the router, I could probably run all my server needs on the router alone, and still have a 800Mb router with hardware NAt, gigabit interfaces, and as-good-as-it-gets Wireless-N.
 

Mike

Member
May 29, 2012
482
16
18
EU
You're obviously buying the wrong routers then...

RT-AC68U with TomatoUSB Shibby, and stick on optware, and all your points are moot. I have a N66U which is pretty old, and I'm running optware on there. All the storage is on a microSD card in the slot inside the router, I could probably run all my server needs on the router alone, and still have a 800Mb router with hardware NAt, gigabit interfaces, and as-good-as-it-gets Wireless-N.
Like I said, it used to be. The N66U is a 3 year old router and the 68U you mention is a dual core ARM model, indeed pretty decent. When I set mine up Asus' catalog consisted of only a bunch of motherboards for the 468 platform.
I've yet to cross paths with a router that has a decent package manager, and can therefor be considered a relative secure option. Also, flashing your typical 150 greenbag router with an unstable firmware packed with proprietary drivers and voiding your warranty is left to a selected few enthusiasts.
 

RyC

Active Member
Oct 17, 2013
359
88
28
My pfsense box (2.1.5) is virtualized on ESXi (5.5) with an Intel NIC passed through for the WAN connection and a vmxnet3 NIC for the LAN connection.
I see a very high CPU usage even with moderate throughput (50-60 mbits). We're talking 100% with 1 vCPU. The weird thing is, though, that the virtual OS doesn't see this. From the pfsense box, CPU usage is reported as being in the 2-10% range by both the pfsense monitor and top. Lots of other people are having the same issues.
There have been other smaller issues that might be due to myself, pfsense or the fact that the pfsense box is virtualized, but the above is 100% due to pfsense being virtualized, and I've been unable to find a fix for it.


Limiting IO will make it a horrible experience when you need to use the box. Everything will slow to a crawl, including the downloads. More resources is the solution IMO.



I'm sure Windows DNS is fine - DHCP not so much in my experience. My point on why I wouldn't do this on Windows in this case is simply consolidation. Why run a Windows machine for DNS and DHCP when you have pfsense doing your routing? Sounds like a waste of 4+ GB vMEM.
I'm running 2.1.5 virtualized in ESXi 5.5 and I don't see the CPU issued you're having. I run 60 mbits throughput regularly and CPU doesn't get close to 100% in pfsense or in ESXi monitoring. I'm running 2 vCPUs however and both LAN and WAN connections are vmxnet3, nothing passed through. Personally I don't notice any performance drawbacks running pfsense virtualized, but maybe I'm not doing anything too taxing.
 

weust

Active Member
Aug 15, 2014
353
44
28
44
Since this topic is completely de-railing, I will give one final answer.

I am not a typical home user. I wouldn't even come here if I were.
My hobby has always been to work with computers. Wanting to know it inside out, do crazy things with them.
Over 13 years ago I made it my work. Work and hobby differ a bit, but for the last 5 years I've been working with VMWare and mainly Hyper-V.
With everything I've done so far, I think I pretty much know what I can and cannot do.

Whether running Windows Servers at home for a home LAN is overkill or not, is besides the point.
It's a hobby. I want to do it because it interests me in setting it up for my small home LAN.
Right now I run a Soekris net6501-30 with pfSense (coming from m0n0wall), a HP Thin client with Linux for DNS/NTP/DHCP and a Mac Mini which serves as the torrent client and iTunes library for my Devialet audio system.
Plus the Synology DS209 which is in progress of being replaced by a DS415Play.

It's all been running in pretty much the same configuration for years.
The Thin client has seen many others before it, from SUN to DEC Alpha to SGI MIPS machines (all running Linux, btw).

The part about virtualizing pfSense clearly lacks experience. Version 2.2 works great on Hyper-V.
Already tested that last September.

The torrent part I can get with heavy I/O. But I don't have that.
Upload is limited to 1MByte/s (configured in the client settings) and the download is a max 21MByte/s.
Right now that's done on a external USB disk connected to the Mac Mini.

I have thought about putting the torrent storage on a separate disk, but I doubt I can put extra storage inside the Supermicro Superserver, and add the extra dual port NIC.
Hence the dual 1TB disks in mirror. And I don't want to use iSCSI connections to the NAS, because I don't want to disks inside the NAS running 24/7. NAS doesn't sleep, disks will. Gotta think about power usage and costs a little bit.
The hobby itself is expensive enough :)

Windows DHCP server is fine. No idea what would be wrong with that.
It's been running here in a Active Directory environment since 2003. Obviously the server OS was updated in the meantime, but other then that it's still pretty much the same configuration.

Hope this explains a bit about MY idea behind my hobby at home.
 

spazoid

Member
Apr 26, 2011
92
10
8
Copenhagen, Denmark
I'm running 2.1.5 virtualized in ESXi 5.5 and I don't see the CPU issued you're having. I run 60 mbits throughput regularly and CPU doesn't get close to 100% in pfsense or in ESXi monitoring. I'm running 2 vCPUs however and both LAN and WAN connections are vmxnet3, nothing passed through. Personally I don't notice any performance drawbacks running pfsense virtualized, but maybe I'm not doing anything too taxing.
Yeah, lots of people do not have this problem, but many others do. Multiple vCPUs wont help, as pfsense 2.1.5 isn't multithreaded. People have tried lots of combinations of vNICs and pNICs and last I checked it looked completely random if you had the issue or not (obviously, in IT nothing is ever random, you just haven't found the root cause of your issues yet).

Weust, it seems you already know what you want to do and how you want to do it, so advice falls on deaf ears and your responses are filled with arrogance and hostility. Good luck with your project.
 
  • Like
Reactions: andrewbedia

weust

Active Member
Aug 15, 2014
353
44
28
44
Spazoid, I know what I will do and how to do it. The topic was purely about the hard drives to use.
Nothing more. The de-railing wasn't my fault?
 

weust

Active Member
Aug 15, 2014
353
44
28
44
Given that a software raid always feels itchy, it might actually be worth going for a SSD for boot and VM storage, and a seperate HDD for the torrent storage.
I use a Samsung EVO 840 Pro in my desktop which is running great.

Might have to think about setting up a backup schedule now too, just in case.
Right now the servers I run aren't backupped at all. Don't feel the need for it as in case of a failure I can switch machines or even re-configure the Airport Extreme to do the routing. Which is now only a Access Point.

Not sure whether I linked the Supermicro model I want yet, but it's [url=
Given that a software raid always feels itchy, it might actually be worth going for a SSD for boot and VM storage, and a seperate HDD for the torrent storage.
I use a Samsung EVO 840 Pro in my desktop which is running great.

Might have to think about setting up a backup schedule now too, just in case.
Right now the servers I run aren't backupped at all. Don't feel the need for it as in case of a failure I can switch machines or even re-configure the Airport Extreme to do the routing. Which is now only a Access Point.

Not sure whether I linked the Supermicro model I want yet, but it's the 5018A-FTN4.
And since I want to put in the dual NIC card, I can't place too many hard drives it seems.
Also, I know there is a Superserver with six NICs, but it doesn't have the NICs on the front side.
Which I really want.
 

nitrobass24

Moderator
Dec 26, 2010
1,087
131
63
TX
So to add to my previous post here is how I have my storage setup for my 2 node hyper-v cluster.

1x 840 Pro 256gb setup as iscsi for my VMs
5x 4tb R5 - Bulk storage, movies, downloads, etc.

Since the SSD is a single disk I have it backed up every night to the R5 array for fast restore. I also backup the backup from the raid5 array to the cloud using crash plan.

This provides me fast disk for my VMs without the cost of multiple disks.
I have a local backup for a fast restore should my SSD die.
I have an offsite backup should my entire storage fail.
 
  • Like
Reactions: PigLover

weust

Active Member
Aug 15, 2014
353
44
28
44
Looks good! What software do you use for backing up the VM's?
Since you need software that knows about VHD(X) files and their state, and all that.
 

Mike

Member
May 29, 2012
482
16
18
EU
So to add to my previous post here is how I have my storage setup for my 2 node hyper-v cluster.

1x 840 Pro 256gb setup as iscsi for my VMs
5x 4tb R5 - Bulk storage, movies, downloads, etc.

Since the SSD is a single disk I have it backed up every night to the R5 array for fast restore. I also backup the backup from the raid5 array to the cloud using crash plan.

This provides me fast disk for my VMs without the cost of multiple disks.
I have a local backup for a fast restore should my SSD die.
I have an offsite backup should my entire storage fail.
Make that an encrypted offsite back-up and you're all set.
 

nitrobass24

Moderator
Dec 26, 2010
1,087
131
63
TX
Looks good! What software do you use for backing up the VM's?
Since you need software that knows about VHD(X) files and their state, and all that.
I am using the built in Lun backup feature in Synology. There is an agent that I had to install on the hyper-v host to take application consistent backups.
 

weust

Active Member
Aug 15, 2014
353
44
28
44
Hadn't read into to it yet, but was hoping it could do that.
Thanks!
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,514
5,806
113
Just got to this thread.

The pair of S3500 800GB drives I got for $300/ ea are now slated for something similar to this. I know they are not 1TB but I am a big replication/ snapshot user and I have a bunch of VMs I just want to have a low power "uh oh" box that I can use a few days a year. Plus, I can use them to add to my growing data set.
 

weust

Active Member
Aug 15, 2014
353
44
28
44
Are your using those drives in a stripe set or a mirror?
I can imagine for snapshots it's great. Replication with 10 or more Gigabit too.
At work I do replication with a 100Mbit connection. Even with compression it will take a 3TB+ VM some time to do the initial replication. It will start at 1am on 1 January.
I don't have the space or time for a export first.