Need to know if I got this right. Passthrough NIC for WAN

[weust]

I would like to know if my thoughts are correct on the following.

Using ESXi, can I use Passthrough of a NIC to a VM running m0n0wall (or whatever router/firewall solution) and have the internet IP address provided by my ISP inside the VM?

To clearify, right now my cable modem is giving my Soekris net6501-30, running m0n0wall, a internet IP address on it's designated WAN port.
The cable model is the basic type. No switch or wireless access point build in.

So for me it would be building a single node ESXi server with several NIC's of which three would be passed through directly to the m0n0wall VM. WAN, LAN and DMZ (well, PlayStation 4 inside a DMZ).
Just too bad the Intel Atom doesn't support vt-d so I need to build a much more power hungry machine.

 

[Entz]

Yes that will work fine.

You can also do it without passthrough (for atoms etc). Simply create a separate vSwitch / port group for a specific pNIC and use that in your VM. What i have always done with my pfSense/RouterOS vm installs. 2550/2750/2758 make great AIW firewall devices imo (router/snort/freepbx etc).

 

[weust]

Good to hear it works like it's in my head

I thought about creating a vSwitch for the pNIC, but I've read several topics on other forums, including pfsense, that without Direct I/O (forgot the VMWare name for it) hits performs in a very bad way.
Also CPU performance was very high, even with multiple vCPU's. And in m0n0walls case, that is single CPU only.

My cable internet connection is 180Mbit down/18Mbit up, and I know I can get that speed easily.
Steam for example shows me downloads of over 20MByte/s. M0n0wall confirms this in it's graph.
So, I need to be sure that with a vSwitch I can reach my speeds, and keep the CPU relativly normal.

What are your connection speeds?

Oh, and what does AIW stand for?

 

[Entz]

AIW = All in one (meaning more then just a firewall, which you would have if you just installed M0n0wall baremetal).

My connection is 100Mbit down/10 up and the guest (RouterOS) only hits 20-25% per core (2) so lots of room there. ESXi is showing a max of 30% on each core and 15% overall for itself (on a C2758), most of which is snort I think. Only time I hit a CPU limit was when I was trying to do IPoIB routing which had issues with getting past 4gbit/s. That being said I haven't tried to find the limit of the gigabit interfaces as it serves me well. I do not see why 180mbit would be an issue provided you are not doing a large number of rules/filters.

Does seem a shame to dedicate a full power cpu just to VT-d, not having pass-through is one thing I dislike about these atoms (the other is they need 1 more damm PCI-e lane so we can have 2 8x slots and BMC) .

 

[weust]

Wouldn't that be AIO then?

You already use more then I do, since m0n0wall doesn't do snort for example.
Seeing as m0n0wall is single-threaded only, and thus doesn't benefit from more then one CPU, looking at what it does for your connection I think I would be safe too.

In my situation I think four ports would suffice.
Dedicated WAN, shared LAN for three machines to a 24 port switch (m0n0wall LAN, VM for DHCP/DNS/NTP and one for a torrent client), one for ESX connectivity. Forgot what I had the fourth port for in mind. Getting late here.
Might rethink things through though. Won't be soon till I buy parts anyway.

 

[Entz]

Yeah your right AIO not AIW, have all in wonders (old ATI cards) on the brain...

 

[weust]

Haha. Those were weird cards. A videocard with onboard TV input.

 

[Entz]

yeah they were, just pulled one out of a system. Was somewhat useful at the time to use your PC as a TV.

 

[weust]

I saw a couple coming from old Mac's over ten years ago.
Never touched them. Too weird for my taste.

 

[weust]

I did some testing last weekend, and sadly I couldn't do a proper full test.

Using a VMSwitch it worked fine. As per your advise. Didn't expect that somehow.
Was expecting not to be able to get the internet IP address on the adapter inside the m0n0wall VM.
Not sure why not, but glad to see it can work.

I also tried vt-d. That wasn't working out at all somehow.
Perhaps because of the used NICs.
The onboard is a Intel i217-something (at work right now, so can't check) and the PCI-E card is a Intel 82547(?) card.

Whatever I tried, I couldn't manage to get an internet IP address on the onboard i217-something card. Using the PCI-E card the VM wouldn't even recognize it. But perhaps that is because the card just isn't supported in FreeBSD 8.3 (m0n0wall 1.8.2).

I did check CPU usage while utilizing the VMSwitch. A speedtest using the ISP's speedtest site showed a CPU usage of close to 2GHz (Intel i7-4770), while doing an actual download of a test file from another ISP's webserver was using a lot more.
Full speed was reached, but it makes me doubt that the Atom 2758 has enough single CPU power for my internet connection.

So far, all test I've read about where using normal desktop CPU's. Not Atoms.
The Atom in my Soekris net6501-30 is only 600MHz, which is a E6xxx model Atom CPU.
M0n0wall runs directly on the hardware, and even there I can see that even a higher speeds increase from my ISP may make the 600MHz too slow to catch up...

 

[Mike]

Use the paravirtual network adapters and you will most likely see that the cpu usage will drop.

 

[weust]

I've seen that term pass by, but didn't look into it.
Will check it out. Thanks.

 

[weust]

I did look at it while configuring last weekend. Problem is that VMXNET3 isn't supported by FreeBSD.
At least, according to the KB article and the information about adapters during the creation of a VM.
Hence I didn't try it out.

Might be worth checking just to be sure.
Have you?

 

[Entz]

Vsphere 5.0+ has FreeBSD drivers/tools available. I have used them to great success in pfSense, so they should work fine in monowall (never tried though). Here is the pfsense thread on the matter.
[How-To] Using VMXNET2/3 NICs in pfSense 2.0

I would expect something similar to work on yours.

Does monowall use a single thread or multiple? Do you have more then one vCPU for the VM? Possible in my case that RouterOS is far more efficient ( it is allocated 4 vCPUs of the 8, 106Mbps is around 30 per core max) and that is using E1000 not the vmxnet ones.

 

[Mike]

Try the newest pfsense releases, they are based on bsd 10 no?

 

[weust]

M0n0wall is single CPU only. No support for SMP build in.
So I only added one vCPU to the VM.

I know I should be able to add drivers to the m0n0wall installation.
Need to look up how to do it though.

I read about the new pfsense, but there isn't a beta available yet?
Or at least not with VMWare support. But using that link I should be able to get it fixed.

What I like about m0n0wall is that it's very clean.
Pfsense is build upon m0n0wall and adds a lot of stuff I don't need or want.
On the other hand, it does support SMP and has official VM support.

Never tried pfsense or RouterOS, but could check them out just in case.

 

[Mike]

Ipfire is (recent) Linux based, might even have Vmware packages through repos.

 

[weust]

Will check that one too. Thanks.

About RouterOS, unless I found the wrong one, isn't this a payed type?
It's just for home use, so I try to my software for free when it comes to the firewall/router.

 

[Entz]

Yeah RouterOS is not free. I like it as I run Mikrotik devices all over the place so having a unified interface from place to place is nice. Definitely go with pfSense or one of the other free ones for at home (or even at work). pfSense is a close favorite of mine, may have more then you need but you do not have to use all the extra stuff if you do not want. It is pretty much bulletproof.

 

[weust]

Yeah, I'm afraid I will have to transition to pfSense someday.
Well, when I finally go virtualized here at home. For now the m0n0wall runs fine.

If I have time, I will try to get pfSense working on my ESXi installation on my workstation.
Need to re-plug drives and install the extra NIC again for that.
Would be interesting to try and get the VMXNET3 working. That would be a must by now.