Nested ESXi Networking Issues

[K D]

I have a small SYS-E200 running a Xeon D-1541 that I take with me when travelling so that I can do VCP practice labs. Since the evaluation period ran out, I had to rebuild the lab and am running into some issues that I've not been able to figure out so far.

Here’s the Setup :

• 1 Physical Host running ESXi 6.5U1. 2 Switches WAN and LAN with pfsense used to route between the 2. WAN is connected to the local network and has internet ACCESS
• LAN has no physical uplink.
• Windows 2016 Core – AD, DNS, DHCP with one vNIC connected to LAN
• OpenFiler, VCenter and a Windows 1 Client VM on the LAN.
• 9 nested ESXi VMs connected to the LAN
• The nested esxi hosts use vSwitch and not dvSwitch.

The Issue :
All the VMs running on the physical host are able to communicate with each other. Any VM deployed on one of the nested ESXi hosts is unable to communicate with any of the VMs on the physical host.

Thoughts Anyone? What information do I need to post to help troubleshoot?

 

[whitey]

Are you using vlan 4095 trick on phys hypervisor and vlan trunking/tagging into the nic that backs your LAN vSwitch? ESXi's using that 4095 port group, phys switch ports tagged for necessary vlans, and using vlan tagging w/ vlans defined on port groups w/in the nested instances? The whole nested inception/networking thing can drive you bonkers to unravel...been there, done that. I'd have to look over my config to be sure but that does spring to mind. Once you get it you will have an AH-HA monent...bet you 'may' be getting bit by promiscuous mode set to reject on the vSwitch security poilcy...seen that a million times. ACCEPT/ACCEPT/ACCEPT across all three sec zones on the vSwitch security settings and things should get better good sir.

This is all from memory, away from lab right now but I can snag screenshots if needed based on vSphere 6.5U1.

 

[K D]

It was Promiscuous mode being set to reject. Changing that to accept fixed it. I forgot about it when I rebuilt the lab. Thanks. All's well now

 

[whitey]

Good deal, figured it was something simple. You using vlan tagging on phys hypervisor host vSwitch phys ports/port groups and mirroring up to nested ESXi vSwitch w/ no vlan or taking the VGT 4095 route and tagging outward (catch-all 4095) on phys ESXi and inward on nested ESXi (defining vlan tags on port groups)?

 

[K D]

Currently tagged the VLANs in the physical host port groups with different vNICs in the nested hosts connected to each portgroup.

 

[whitey]

Sweet, have fun/good luck w/ cert prep!

 

[K D]

Thanks for the assist.