FreeNAS Corral and bhyve network question

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

strh

New Member
Mar 2, 2015
20
6
3
I installed Corral and have been playing around with VMs.

I've observed an interesting problem when running some simple network tests and I'm wondering if anyone has seen similar results in their testing.

All tests are running on a 10G Network and involve running the following 2 commands.
  1. On FreeNAS Corral Host: iperf -s
  2. On a remote OS targeting above host: iperf -c XXX.XXX.XXX.XXX -P1 -i1 -t100
Test 1:
  • Result: 9.41 Gbits/sec
So far so good.

Test 2:
Next, start a single VM (any VM) with a bridged network mode device (same network - no vlan) on the FreeNAS host and run the same test (no direct interaction with the VM)
  • Result: ~3.2 Gbits/sec

Test 3:
Next, shutdown the VM with the bridged network mode device (same network - no vlan) on the FreeNAS host and run the same test.
  • Result: 9.41 Gbits/sec

Test 4:
Next, start a VM with a NAT network mode device on the FreeNAS host and run the same test.
  • Result: 9.41 Gbits/sec

Current conclusion? Bridged VMs severely impact network performance on the host?

Some further notes:
I called out no-vlan above because initial testing is suggesting I can't get back to the full wire-speed once I've started a VM with a bridged vlan enabled network. Shutting down the VM will still result in poor iperf performance numbers until i reboot the Host machine.

I'm also seeing poor VM to VM network performance using bridged vlan enabled network devices, which might not be surprising given the above results.

Cheers.
 
  • Like
Reactions: gigatexal

strh

New Member
Mar 2, 2015
20
6
3
Or MTU size on the test 2 vlan anyway to check if that's at 9k

I'm not a networking expert so im spitballing ideas
Everything is all MTU 1500 at the moment. Also, vlan config was not part of test 2 as I attempted to keep it simple initially.

Cheers.
 

strh

New Member
Mar 2, 2015
20
6
3
Some additional results:

Test 5:
  1. On FreeNAS Corral Host: iperf -s
  2. On the NAT network mode VM guest targeting its host(above): iperf -c XXX.XXX.XXX.XXX -P1 -i1 -t100
  3. Result: 2.2 Gbits/s

Test 6:
  1. On a remote FreeNAS 9.10U2 box (over 10Gbe): iperf -s
  2. On the NAT network mode VM guest hosted on the FreeNAS Corral: iperf -c XXX.XXX.XXX.XXX -P1 -i1 -t100
  3. Result: 4.0 Gbits/s
I'll stop posting now and try figure out if the above results are to be expected on a FreeNAS box. Hope the data is useful.

Cheers.
 
  • Like
Reactions: gigatexal