Hello, for reference this is regarding iSCSI between a FreeNAS VM and a Server 2019 VM.
I can think of a multitude of ways to connect the two to each other, and I was wondering, does it matter?
A: vSwitch with no physical interfaces and the two VMs connected with vmxnet3 adapters
B: Same as A, but associated with a physical NIC (and say it's a nicer server NIC with offload capabilities)
C: NICs passed-through to each VM and connected externally with a cross-over cable
D: SRIOV with each VM assigned a VF off of one NIC
- In case A, this should be efficient because as I understand it, with paravirtual NICs the hypervisor will do a memory copy
- In case D, should be similar to case C, although it appears to me that people have had some stability issues with SRIOV
I know there are some more corner cases, as well as many little idiosyncrasies that could vary in how much of an effect they have.
So, just wondering what people on this forum like to do, and why?
For my own answer to this question, here are some of my considerations:
- I tend to favor passing the NIC through at least to FreeNAS, just like I do with the HBA, so it mostly feels like a physical machine
- I'm afraid of SRIOV, I think it's really cool, but I have no experience with it
- I remember hearing something in the past that VMware can be limited to around 4Gig/sec, and that it is a driver/CPU limitation, but it seems maybe that has been solved / surpassed for some time.
Thanks
-JCL
I can think of a multitude of ways to connect the two to each other, and I was wondering, does it matter?
A: vSwitch with no physical interfaces and the two VMs connected with vmxnet3 adapters
B: Same as A, but associated with a physical NIC (and say it's a nicer server NIC with offload capabilities)
C: NICs passed-through to each VM and connected externally with a cross-over cable
D: SRIOV with each VM assigned a VF off of one NIC
- In case A, this should be efficient because as I understand it, with paravirtual NICs the hypervisor will do a memory copy
- If you used old emulated NICs, you would suffer higher CPU, worse performance, probably both
- In case B, would this make any difference over case A at all? Maybe not perform better, but use less CPU?- And, how would the speed of the NIC play into it? Would it? 1/10/25/40/50/100/etc.
- In case C, I would expect very low CPU as you are bypassing the hypervisor now, and it would be down to the drivers in each OS- However, you are going all the way down the network stack and back again
- In case D, should be similar to case C, although it appears to me that people have had some stability issues with SRIOV
I know there are some more corner cases, as well as many little idiosyncrasies that could vary in how much of an effect they have.
So, just wondering what people on this forum like to do, and why?
For my own answer to this question, here are some of my considerations:
- I tend to favor passing the NIC through at least to FreeNAS, just like I do with the HBA, so it mostly feels like a physical machine
- Sort of my take on the do/don't virtualize FreeNAS
- Otherwise, maybe just option A or B and keep it simple
- For the Windows machine, I am much more comfortable trying any of these options, because it is the OS I am most comfortable with- I'm afraid of SRIOV, I think it's really cool, but I have no experience with it
- I remember hearing something in the past that VMware can be limited to around 4Gig/sec, and that it is a driver/CPU limitation, but it seems maybe that has been solved / surpassed for some time.
Thanks
-JCL
