STP behaviour question

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
Hi,

clearly no networking expert here - so maybe somebody with more experience in this can answer :)

I am trying to get more resilience in my network by adding a second cross building interconnect. The simplified target design would basically look like this:
upload_2019-5-19_10-47-46.png
Now from what I understand this would be a case for STP (or RVSTP[+]), but its not working as expected.

Whenever i enable the second connection from the '36 to a 350x I get link flapping (i.e. one of the interfaces gets shut down every couple of seconds, then its up again, then down and so on).

I have set stp priorities across all switches assuming that should enable a least cost route but its not working as expected yet.

I have not dabbled with any of the other settings (loop guard, port types etc) as when i originally activated loop guard it shut down my access port to the switch and I had some fun times getting everything going again, so not too keen on 'uneducated playing around'.

I am sure I must be missing something fairly simple here, but I just have not the background to see it.

The goal would be to have all links up&running all the time providing resilience. If I also could get load distribution /least cost routing at the same time then that would be a nice bonus. I am not looking to separate vlans on switches atm, since for failover reasons all vlans might need to cross buildings too.

Thanks:)
 

azev

Active Member
Jan 18, 2013
757
236
43
Lets assume you want the 6036 to be the root of your spanning tree then all you have to do is as follows
in a pure cisco environment set the lowest spanning-tree priority on the 6036 and then a higher priority value on both the sg350x.
However with non cisco switches in the mix you might have to play around a bit with the spanning tree setup on the non cisco box.
 

itronin

Well-Known Member
Nov 24, 2018
924
590
93
Denver, Colorado
Are the SG350x's in a stack?
If so then...
I believe the SG350x supports cross stack LAG.
I have no familiarity with the SX6036 - but am guessing it probably also supports LAG...
If the SG350x's are not in a stack do you want to stack them?

thinking out loud - then again maybe you want to play with STP as much as providing resiliency.
 

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
Lets assume you want the 6036 to be the root of your spanning tree then all you have to do is as follows
in a pure cisco environment set the lowest spanning-tree priority on the 6036 and then a higher priority value on both the sg350x.
However with non cisco switches in the mix you might have to play around a bit with the spanning tree setup on the non cisco box.
Well thats basically what I did before - defining a root and expecting things to work, but that didnt do it;)

Are the SG350x's in a stack?
If so then...
I believe the SG350x supports cross stack LAG.
I have no familiarity with the SX6036 - but am guessing it probably also supports LAG...
If the SG350x's are not in a stack do you want to stack them?
No atm they are not stacked - but that would be an option that probably would resolve the specific issue.
However I wonder why its not working the way I expected it to work (i.e. is it not working as supposed or are my expectations wrong;))
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
From all I read they should be downward compatible, but yes.

sx60361 [standalone: master] (config) # show spanning-tree
Switch : ethernet-default
Spanning tree protocol rpvst: enabled
Spanning tree force version : 2

upload_2019-5-19_23-23-51.png

upload_2019-5-19_23-24-38.png
 

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
Ok, so if should work and does not then let's explore alternatives.

Stacking the 350's might be one, but I'll need to check if that works with a single connection, i think to remember that cisco stacking needed two stack ports which means I can't do it while having the second connection as uplink to the mlx.

Any other ideas? :)
 

istamov

New Member
Jul 31, 2015
15
5
3
Can you try with MSTP on all devices?
If I find some time tonight I can try this at home and will report back.

Configuration on the SX6036 should be something similar to (according to this archived document):

Code:
switch (config)# interface ethernet 1/1 switchport mode trunk
switch (config)# interface ethernet 1/2 switchport mode trunk

switch (config)# spanning-tree mode mst
switch (config)# spanning-tree mst name mstp
  ### MSTP revision must be the same throughout the MSTP subnet
switch (config)# spanning-tree mst revision 1
  ### Add VLANs to an MSTP instance
switch (config)# spanning-tree mst 10 vlan 101
switch (config)# spanning-tree mst 10 vlan 102
switch (config)# spanning-tree mst 10 vlan 103
switch (config)# spanning-tree mst 10 vlan 104
  ### Set this switch as the root bridge for MST instance 10
switch (config)# spanning-tree mst 10 priority 0
 

itronin

Well-Known Member
Nov 24, 2018
924
590
93
Denver, Colorado
Ok, so if should work and does not then let's explore alternatives.

Stacking the 350's might be one, but I'll need to check if that works with a single connection, i think to remember that cisco stacking needed two stack ports which means I can't do it while having the second connection as uplink to the mlx.

Any other ideas? :)
For the SG500X and SG350X series, you can stack with a single cable interconnect between switches. Its called "chain" versus "ring". The stack should be auto configured to type during the stacking discovery phase at boot. With 2 switches, one will be the master and the other backup master. Obviously the risk is greater of a split brain scenario with a cross stack LAG if the stack connection itself fails but the connections to the the SX switch do not.

see Configure Stack Settings on an SG350X Switch
 

istamov

New Member
Jul 31, 2015
15
5
3
Okay, so I did a quick test setup and it seems to be working.
I used 1 SX6036 and 2 Cisco C3750E switches, all having VLAN 222 tagged and connected in between as follows:
Code:
C3750E-A, Te1/0/1 => C3750E-B, Te1/0/1
SX6036, Eth1/1 => C3750E-A, Te1/0/2
SX6036, Eth1/2 => C3750E-B, Te1/0/2
Here is the working configuration that I used:
Code:
### SX6036
vlan 222
interface ethernet 1/1 switchport mode trunk
interface ethernet 1/2 switchport mode trunk
spanning-tree mode mst
spanning-tree mst 10 vlan 222
spanning-tree mst 10 priority 0
spanning-tree mst name mstp
spanning-tree mst revision 1

### C3750E-A
vlan 222
interface TenGigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 222
 switchport mode trunk
exit
interface TenGigabitEthernet1/0/2
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 222
 switchport mode trunk
exit
spanning-tree mode mst
spanning-tree loopguard default
spanning-tree logging
spanning-tree extend system-id
spanning-tree pathcost method long
spanning-tree mst 10 priority 4096
spanning-tree vlan 222 priority 4096
spanning-tree vlan 222 hello-time 3
spanning-tree vlan 222 forward-time 16
spanning-tree vlan 222 max-age 15

### C3750E-B
vlan 222
interface TenGigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 222
 switchport mode trunk
exit
interface TenGigabitEthernet1/0/2
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 222
 switchport mode trunk
exit
spanning-tree mode mst
spanning-tree loopguard default
spanning-tree logging
spanning-tree extend system-id
spanning-tree pathcost method long
spanning-tree mst 10 priority 8192
spanning-tree vlan 222 priority 8192
spanning-tree vlan 222 hello-time 3
spanning-tree vlan 222 forward-time 16
spanning-tree vlan 222 max-age 15
Thus making SX6036 the root bridge.
So when all ports are up, the root bridge is reachable via C3750E-A, Te1/0/2 and if that port is shutdown (link to the root is lost), then the root path becomes via C3750E-B, Te1/0/2.

Code:
C3750E-A#show spanning-tree

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     0002.c9e3.9a20
             Cost        2000
             Port        30 (TenGigabitEthernet1/0/2)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     0021.d80e.3800
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/0/1             Desg FWD 2000      128.29   P2p
Te1/0/2             Root FWD 2000      128.30   P2p Bound(RSTP)

C3750E-A#
C3750E-A#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
C3750E-A(config)#int Te1/0/2
C3750E-A(config-if)#shu
C3750E-A(config-if)#shutdown
C3750E-A(config-if)#end
C3750E-A#
C3750E-A#show spanning-tree

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     0002.c9e3.9a20
             Cost        2000
             Port        29 (TenGigabitEthernet1/0/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     0021.d80e.3800
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/0/1             Root FWD 2000      128.29   P2p
Code:
C3750E-B#show spanning-tree

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     0002.c9e3.9a20
             Cost        2000
             Port        29 (TenGigabitEthernet1/0/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     d4a0.2a06.7700
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/0/1             Root FWD 2000      128.29   P2p
Te1/0/2             Altn BLK 2000      128.30   P2p Bound(RSTP)


C3750E-B#
*Jan  2 00:21:30.989: %SPANTREE-5-ROOTCHANGE: Root Changed for instance 0: New Root Port is TenGigabitEthernet1/0/2. New Root Mac Address is 0002.c9e3.9a20
*Jan  2 00:21:30.998: %SPANTREE-5-TOPOTRAP: Topology Change Trap for instance 0
C3750E-B#
C3750E-B#
C3750E-B#
C3750E-B#
C3750E-B#show spanning-tree

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     0002.c9e3.9a20
             Cost        2000
             Port        30 (TenGigabitEthernet1/0/2)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     d4a0.2a06.7700
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/0/1             Desg FWD 2000      128.29   P2p
Te1/0/2             Root FWD 2000      128.30   P2p Bound(RSTP)
Code:
switch-sx6036 [standalone: master] # show spanning-tree

Switch                     : ethernet-default
Spanning tree protocol mst : enabled
Spanning tree force version: 3

Root ID:
  Priority           : 32768
  Address            : 00:02:c9:e3:9a:20  
  Bridge root        : yes
  Hello Time (sec)   : 2
  Max Age (sec)      : 20
  Forward Delay (sec): 15

  MST00:
    Bridge is executing the mstp compatible Spanning Tree Protocol

Bridge ID:
  Priority           : 32768
  Address            : 00:02:c9:e3:9a:20  
  Hello Time (sec)   : 2
  Max Age (sec)      : 20
  Forward Delay (sec): 15

L: Loop Inconsistent
R: Root Inconsistent
G: BPDU Guard Inconsistent

--------------------------------------------------------------------------
Interface         Role         Sts              Cost      Prio   Type    
--------------------------------------------------------------------------
Eth1/1            Designated   Forwarding       2000      128    normal  
Eth1/2            Designated   Forwarding       2000      128    normal
Please excuse me that I am not going into more detail at the moment, if you need any other information or tests, just ask and I will try to reply when I can :)
 

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
Many many thanks for this, I'll give it a try and will let you know how it worked out:)
 

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
I think I found an error in my config - Cisco loopback detection was active in addition to STP.
So when ever i activated the second port the cisco loopback detection (independent of stp) kicked in and turned off one of the ports.
The other one got turned of due to a 'port type inconsistency' (what ever that is). That also vanished when i disabled the extra loopback detection,
Running with rpvst atm and switched root to Mellanox as suggested.

Any reason to have one box root over any other?
 

Rand__

Well-Known Member
Mar 6, 2014
6,178
1,532
113
So my issue is back - intermittently...
The last big one was in May which was when I opened this thread ... since then it happens every now and then and more frequently again this week

upload_2019-7-23_17-53-28.png



What happens is that my ZeroClient gets a disconnect and after reconnecting everything is working fine... What I see is that the switches got a link down event but I can't identify why.
Both switches are on debug mode logging, but I can't see anything causing this issue ...
It seems to originate on switch G ( occurred 2 secs earlier) but no mentioning of why - does anyone know how to see that?



Code:
Switch K

2147460585     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: te1/0/1 of vlan 18: STP status

Forwarding   
2147460586     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/24 of vlan 18: STP status

Blocking   
2147460587     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/20 of vlan 18: STP status

Blocking   
2147460588     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/12 of vlan 18: STP status

Blocking   
2147460589     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/10 of vlan 18: STP status

Blocking   
2147460590     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/5 of vlan 18: STP status

Blocking   
2147460591     2019-Jul-23 17:36:09    Warning     %STP-W-PORTSTATUS: gi1/0/1 of vlan 18: STP status

Blocking   
2147460592     2019-Jul-23 17:36:09    Informational     %LINK-I-Up:  te1/0/1   
2147460593     2019-Jul-23 17:36:05    Warning     %LINK-W-Down:  te1/0/1   


Switch G

2147481180     2019-Jul-23 17:36:11    Warning     %STP-W-PORTSTATUS: te1/0/1 of vlan 9: STP status

Forwarding   
2147481181     2019-Jul-23 17:36:11    Warning     %STP-W-PORTSTATUS: te1/0/1 of vlan 2: STP status

Forwarding   
2147481182     2019-Jul-23 17:36:11    Warning     %STP-W-PORTSTATUS: te1/0/1 of vlan 7: STP status

Forwarding   
2147481183     2019-Jul-23 17:36:11    Warning     %STP-W-PORTSTATUS: te1/0/1 of vlan 18: STP status

Forwarding   
2147481184     2019-Jul-23 17:36:10    Informational     %LINK-I-Up:  te1/0/1   
2147481185     2019-Jul-23 17:36:07    Warning     %LINK-W-Down:  te1/0/1
Not sure if a portmirror of that port and a trace would help ? Difficult now since its (thankfully) happening every couple of hours now instead of every few (5) minutes as it used to ... So its just annoying and I'd like to get rid of it but not spend too much time on it ;)

Any ideas?