Direct connection between NAS and ESXi?

[spyrule]

I'm wondering if someone can point me on how to setup a direct fiber connection between my ESXi 5.5 box and my NAS (Synology box with a 10Gig connection). I have physical connectivity already, I'm just trying to setup the connection so that all of my VMs storage uses the 10Gig, and then have all of the VM's IP traffic use my standard 1Gig ports.

Right now, everything is sharing two 1Gig ports for all traffic types, but splitting the two would be awesome.

 

[spyrule]

anybody?

 

[marv]

just set up ip addresses on 10g interfaces and on esxi change ip address for your iscsi/nfs datastores to your nas box's 10g interface ip. that should be it.

 

[spyrule]

Thanks, trying this now

 

[spyrule]

So the only way I could get this to work was to set my NAS's IP as 192.168.3.1 and set itself as a gateway. Only after doing that would my ESXi setup see the connection (despite it being a direct connection and on the same subnet). Odd.

 

[spyrule]

Ok, turns out it wasn't the gateway, but the MTU 9k setting. Set it back to 1500, and changed the ip to .3.2 - and 3.3 respectively, and changed the gateway to my actual gateway (even though they cant reach it), now it works.

Its odd though that I cannot set both cards to 9K MTU despite both cards being the same brand/type.

 

[Deci]

you have a vmkernel port on your 1g nic, and another seperate vswitch with a vmkernel port for your storage connected to the 10gb nic?

the 10g should be on a completely different subnet, ie if you are using 192.168.3.xxx/24 (255.255.255.0) for your normal 1g stuff, set your 10g to 192.168.4.1 & 192.168.4.2 /30 (255.255.255.252) however you can have what ever ip range you want if thats not avaliable, you dont enter a gateway for those interfaces, just the IP and subnet mask, gateway is only required for routing traffic beyond the local subnet, that link doesnt have any external connectivity if its a direct connection and multiple gateways can cause issues.

 

[spyrule]

yeah, I ended up figuring it out, I simply hadn't done it in forever and forgot how. ><
I now have 3 vSwitches; 0 = Primary VMs (1x1Gig), 1 = 10Gig storage, 2 = Development/extraneous VMs (1x1Gig) .Everything seems to be working as expected. I'm just a little perplexed why I can't do 9K MTU on both 10Gig nics, but I'm not too worried about it for now.

In a few months, both 10Gigs are going to get connected to a switch instead, and I'll be using them for standard traffic as well. Or I might sell both single port NICs for two Dual ports, and keep the direct connection as well as one 10Gig for standard network traffic for both Storage and ESXi box.

Thanks for the help anyway.

 

[vBuild2BAU]

yeah, I ended up figuring it out, I simply hadn't done it in forever and forgot how. ><
I now have 3 vSwitches; 0 = Primary VMs (1x1Gig), 1 = 10Gig storage, 2 = Development/extraneous VMs (1x1Gig) .Everything seems to be working as expected. I'm just a little perplexed why I can't do 9K MTU on both 10Gig nics, but I'm not too worried about it for now.

In a few months, both 10Gigs are going to get connected to a switch instead, and I'll be using them for standard traffic as well. Or I might sell both single port NICs for two Dual ports, and keep the direct connection as well as one 10Gig for standard network traffic for both Storage and ESXi box.

Thanks for the help anyway.

Jumbo frames have to be set all the way through. On the VMkernel on the vSwitch on the physical switch and on the Synology.

Did you set Jumbo frames on the VMkernel on vSwitch1?
Did you set Jumbo Frames on the vSwitch1?

https://kb.vmware.com/selfservice/m...nguage=en_US&cmd=displayKC&externalId=1007654

 

[spyrule]

Yeah, I set the 9K MTU on both my synology NAS, as well as the vSwitch associated with the 10Gig interface. I don't have a switch between the two since its a direct connection. I'm suspecting the fragmentation was too high, because I could get a basic connection, but I couldn't browse the datastore once connected (I'd get no results). Unfortunately, I don't have a lot of time/oppertunity to spend troubleshooting the issue, so I'll leave it at 1500 for now. I may test 4 or 6k jumbo frames and see what happens on my next maintenance window.

 

[idonbuckley]

hey, thanks for this you guys. I need to get the info about the dedicated subnet for just the iSCSI traffic.

so fyi my process was -

Setup the vmkernel NIC with 192.168.10.1/255.255.255.252
adding it to its own dedicated port group "iSCSI" and "vSwitch_iSCSI"
added the vmnic3 (in this case), the nic I was plugging the NAS into, into the same "vSwitch_iSCSI"

setup the NAS with the port as 192.168.10.2 / 255.255.255.252

Plugged the NAS into the vmnic on the server (link up)

Setup the software iSCSI to do discovery on the 192.168.10.2 target, and it finds the target on the NAS right away.

Saludos!

 

[dswartz]

Nice necro, bro!