Mellanox SX6036 L3 VLAN woes

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Castlers

New Member
Feb 22, 2021
23
1
3
I'm having issues routing from the switch to the router/ISP. I have 2 VLANs created to separate the Workstations and the Servers, and while Inter-VLAN routing is working, getting out of the switch and to the router is not working. I cannot create a routerport on this switch as it says the device doesn't support it for some reason and whenever you try to create a route you cannot specify an interface so it will automatically pick one for you depending on the gateway you put in.

IP Routing is enabled on this Switch

VLAN 10 is a 10.0.10.1/255.255.255.0
VLAN 20 is a 10.0.20.1/255.255.255.0

chrome_Di0u1GY5no.pngchrome_PWShWQsapo.pngchrome_w4SbT4zaEY.png


Network Topology.png

In the VLAN table Eth1/1 is the port that goes from the SX6036 to the MikroTik switch and that's what I wanted to originally make a routerport so it would be easier to do all of this, but does anyone have experience with Mellanox switches and where I can go from here to actually make this function?

Making routes from any of the VLANS IP (10.0.10.0) out on 0.0.0.0 or 10.0.0.0 won't attach it to the right interface and the route won't be active.|
The MikroTik has no VLANs on it so it is flat.
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,847
1,223
113
I'd guess the problem is that your management interface is in the subnet that goes to the router and you don't have a router interface in that subnet

The best way would probably be to move the management interface to a different subnet and create a transit VLAN

e.g.
Move mgmt0 to a different subnet, e.g. 10.123.123.123/24
Add new (transit) VLAN 999
Set VLAN 999 as native / untagged VLAN on Eth1/1
Add router interface for VLAN 999, new IP (like 10.0.0.99/24)

it should then automatically set the default route correctly (going through VLAN 999)

P.S. You should always use the CLI for configuration on enterprise switches, the Web UIs suck :)
 
  • Like
Reactions: Aluminat

Castlers

New Member
Feb 22, 2021
23
1
3
IP Interface.PNGIP Route.PNGVLAN.PNG
It does look a lot cleaner after moving the mgmt0 off this current subnet and it looks like the route from vlan999 is actually correct, but I cannot get past it to reach 10.0.0.1

Both sides can ping 10.0.0.99 though, which is good I guess. I went ahead and put a static route on my router and that didn't do anything eitherStatic Route.PNG
Also I fully agree with configuring on CLI for these switches. This just made better pictures and sometimes I feel doing VLANs is better, but I try and do routes on CLI because it is miserable on this web gui.

Do you have any other things I could try with this?
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,847
1,223
113
You're missing the default route in the SX6036. You need to add the default route (dest 0.0.0.0/0) via 10.0.0.1 in the switch

You also need to make your ASUS router aware of the workstations (i.e. add static routes to 10.0.10.0/24 and 10.0.20.0/24 via 10.0.0.99)
 
  • Like
Reactions: Castlers

Castlers

New Member
Feb 22, 2021
23
1
3
Alright it is working now! I set the static routes on both the router for the VLANs and the in the CLI set the route from VLAN 999 to the router. Only issue is now the speeds are awful.
VLAN final.PNGIP Interface Final.PNGIP Routes Final.PNGRouter Routes Final.PNG