Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[kemic]

Which Switch ? Show us the config so we can see the ports you are using ? Dynamic or static LACP - who/what is on the other end and what are they configured for ? It does sound like STP though

Do you have router firmware or switch firmware on the unit ?

Craig

Thanks for the reply.

Switch is an ICX 6610 with router FW. Ports I'm using are 1/2/2 and 1/2/3. Dynamic LACP. Latest stable version of TrueNAS core on the other end, went thru the prompts to create the lag on the TrueNAS and everything is all good only had to reboot the switch to make the other port come up. I suspect STP like I said or just general weirdness with using the breakout cables on the rear ports? I can't seem to make speed-duplex 10g-full stick on 1/2/2 either. Command doesn't produce any error but the config doesn't show like the other ports. No issues setting up the other LACP lag for my Cisco switch either, it's just using the 1Gbe copper ports on the front though...

Relavent Config Sections:

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
!
global-stp
!
!
lag cisco2960 dynamic id 1
 ports ethernet 1/1/45 to 1/1/46
 primary-port 1/1/45
 lacp-timeout short
 deploy
!
lag testing dynamic id 2
 ports ethernet 1/2/2 to 1/2/3
 primary-port 1/2/2
 lacp-timeout short
 deploy
...
vlan 13 name Management by port
 tagged ethe 1/1/45 to 1/1/46 ethe 1/3/1 to 1/3/4
 untagged ethe 1/1/1 to 1/1/2 ethe 1/2/2 to 1/2/3
 router-interface ve 13
 spanning-tree
...
interface ethernet 1/1/45
 dual-mode  10
!
interface ethernet 1/3/1
 speed-duplex 10G-full
!
interface ethernet 1/3/2
 speed-duplex 10G-full
!
interface ethernet 1/3/3
 speed-duplex 10G-full
!
interface ethernet 1/3/4
 speed-duplex 10G-full
!
interface ethernet 1/3/5
 speed-duplex 10G-full
!
interface ethernet 1/3/6
 speed-duplex 10G-full
!
interface ethernet 1/3/7
 speed-duplex 10G-full
!
interface ethernet 1/3/8
 speed-duplex 10G-full
!
...

 

[Craig Curtin]

Thanks for the reply.

Switch is an ICX 6610 with router FW. Ports I'm using are 1/2/2 and 1/2/3. Dynamic LACP. Latest stable version of TrueNAS core on the other end, went thru the prompts to create the lag on the TrueNAS and everything is all good only had to reboot the switch to make the other port come up. I suspect STP like I said or just general weirdness with using the breakout cables on the rear ports? I can't seem to make speed-duplex 10g-full stick on 1/2/2 either. Command doesn't produce any error but the config doesn't show like the other ports. No issues setting up the other LACP lag for my Cisco switch either, it's just using the 1Gbe copper ports on the front though...

Relavent Config Sections:

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
!
global-stp
!
!
lag cisco2960 dynamic id 1
ports ethernet 1/1/45 to 1/1/46
primary-port 1/1/45
lacp-timeout short
deploy
!
lag testing dynamic id 2
ports ethernet 1/2/2 to 1/2/3
primary-port 1/2/2
lacp-timeout short
deploy
...
vlan 13 name Management by port
tagged ethe 1/1/45 to 1/1/46 ethe 1/3/1 to 1/3/4
untagged ethe 1/1/1 to 1/1/2 ethe 1/2/2 to 1/2/3
router-interface ve 13
spanning-tree
...
interface ethernet 1/1/45
dual-mode  10
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
...

1) The Speed/Duplex settings are meaningless on the 1/2/1 to 1/2/10 ports so they accept the commands but do nothing with them (and do not store in the config - so you are good there

2) Dynamic LACP is a bit hit and miss between vendors - maybe go for static and see how you go

3) You should have Stack Disable in your config at the end of the 3 lines on the initial config - this is mine

SSH@6610-Basement#show run
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable

4) Have a read back a couple of pages to some stuff i have posted about issues with my ESXI and 10GB - worth reading through as a couple of the suggestions were to do with Global STP, VRE routing and the like.

Craig

 

[AndroidCat]

Thanks for the reply.

Switch is an ICX 6610 with router FW. Ports I'm using are 1/2/2 and 1/2/3. Dynamic LACP. Latest stable version of TrueNAS core on the other end, went thru the prompts to create the lag on the TrueNAS and everything is all good only had to reboot the switch to make the other port come up. I suspect STP like I said or just general weirdness with using the breakout cables on the rear ports? I can't seem to make speed-duplex 10g-full stick on 1/2/2 either. Command doesn't produce any error but the config doesn't show like the other ports. No issues setting up the other LACP lag for my Cisco switch either, it's just using the 1Gbe copper ports on the front though...

...

Between a switch and TrueNAS -
I'd stay away from LAG config, where both link are active at the same time. Instead one link should be active and the other standby for redundancy. Both active never worked well for me (I think they also discourage this somewhere in docs).

 

[thebwack]

Before I go through the manual and reset and config my 7250 I'm curious how it will play with my two 6610 I've already deployed. Mainly wanting to setup a LAG between my 6610 rear breakout and a 7250's SFP ports. The 6610 is on 8.0.30 and the 7250 can go up to 8.0.95. Will these different series and FastIron releases work well together? Getting my serial cable tomorrow. Thanks!

 

[vangoose]

Before I go through the manual and reset and config my 7250 I'm curious how it will play with my two 6610 I've already deployed. Mainly wanting to setup a LAG between my 6610 rear breakout and a 7250's SFP ports. The 6610 is on 8.0.30 and the 7250 can go up to 8.0.95. Will these different series and FastIron releases work well together? Getting my serial cable tomorrow. Thanks!

I have lacp between ICX and Arista, and between ICX and Juniper, and between Arista and Netgear. Different version shouldn't matter at all.

 

[LodeRunner]

Before I go through the manual and reset and config my 7250 I'm curious how it will play with my two 6610 I've already deployed. Mainly wanting to setup a LAG between my 6610 rear breakout and a 7250's SFP ports. The 6610 is on 8.0.30 and the 7250 can go up to 8.0.95. Will these different series and FastIron releases work well together? Getting my serial cable tomorrow. Thanks!

As long as both sides of the LAG are the same mode, you shouldn't have a problem. Like vangoose, I have an ICX connected to an Arista using LACP; no issues.

 

[thebwack]

Awesome, makes sense. Thanks

 

[Craig Curtin]

As long as both sides of the LAG are the same mode, you shouldn't have a problem. Like vangoose, I have an ICX connected to an Arista using LACP; no issues.

I guess an extension of this question - if i have 2 x 6450 - do i just connect them together through the front SFP+ ports with a DAC and all done ?

Craig

 

[LodeRunner]

I guess an extension of this question - if i have 2 x 6450 - do i just connect them together through the front SFP+ ports with a DAC and all done ?

Craig

You should configure the ports first, otherwise you'd have spanning tree will down a port or you forgot about spanning tree and you have a loop.

Or down the ports, cable them, configure, then enable them.

I don't know if the 6450 has any automatic stacking; the 7450's I deployed at work automatically stacked using the 40G ports, but I connected them in a linear stack first, waited for it to configure, then closed the loop.

Here're some config snippets from my 7450 at home using both 40G ports as a LACP bundle to my Arista core:

lag 40G-Core dynamic id 4
ports ethe 1/3/1 ethe 1/4/1
port-name 40G-Core:E35 ethernet 1/3/1
port-name 40G-Core:E36 ethernet 1/4/1
!
...
interface lag 4
spanning-tree 802-1w admin-pt2pt-mac
!

VLANs are assigned to it the same as any other port, referencing the LAG port (in this case lg4 for me) rather than the physical ports.

Here's the Arista side, which is more or less Cisco syntax:

interface Port-Channel1
   description 1G Core LAG
   switchport trunk allowed vlan <range>
   switchport mode trunk
   port-channel min-links 1
!
...
interface Ethernet35
   description To Core:1/3/1
   switchport mode trunk
   channel-group 1 mode active
!
interface Ethernet36
   description To Core:1/4/1
   switchport mode trunk
   channel-group 1 mode active
!

 

[Craig Curtin]

You should configure the ports first, otherwise you'd have spanning tree will down a port or you forgot about spanning tree and you have a loop.

Or down the ports, cable them, configure, then enable them.

I don't know if the 6450 has any automatic stacking; the 7450's I deployed at work automatically stacked using the 40G ports, but I connected them in a linear stack first, waited for it to configure, then closed the loop.

Here're some config snippets from my 7450 at home using both 40G ports as a LACP bundle to my Arista core:

lag 40G-Core dynamic id 4
ports ethe 1/3/1 ethe 1/4/1
port-name 40G-Core:E35 ethernet 1/3/1
port-name 40G-Core:E36 ethernet 1/4/1
!
...
interface lag 4
spanning-tree 802-1w admin-pt2pt-mac
!

VLANs are assigned to it the same as any other port, referencing the LAG port (in this case lg4 for me) rather than the physical ports.

Here's the Arista side, which is more or less Cisco syntax:

interface Port-Channel1
   description 1G Core LAG
   switchport trunk allowed vlan <range>
   switchport mode trunk
   port-channel min-links 1
!
...
interface Ethernet35
   description To Core:1/3/1
   switchport mode trunk
   channel-group 1 mode active
!
interface Ethernet36
   description To Core:1/4/1
   switchport mode trunk
   channel-group 1 mode active
!

Cheers mate - much appreciated.

No stacking ports per se on the 6450 - i think they are really designed to be closet switches on the edge of the network - cabled back through 10GB to a central 6610 etc

Craig

 

[LodeRunner]

Cheers mate - much appreciated.

No stacking ports per se on the 6450 - i think they are really designed to be closet switches on the edge of the network - cabled back through 10GB to a central 6610 etc

Craig

The ICX family doesn't have dedicated stacking ports, but the configuration has some presets. The 6450 for example: Brocade ICX 6610 Stackable Switch Hardware Installation Guide - Manual (Page 46)

 

[Craig Curtin]

The ICX family doesn't have dedicated stacking ports, but the configuration has some presets. The 6450 for example: Brocade ICX 6610 Stackable Switch Hardware Installation Guide - Manual (Page 46)

thanks - will follow that and your advice above

Craig

 

[jkieffer72]

Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...

 

[jkieffer72]

Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...

I was reading thru this thread and saw a few suggestions:

Reboot windows. And/or uninstall/reinstall the com port drivers.
Try a real serial port, not a USB converter.

 

[LodeRunner]

I see that from my APC PDU's and a USB serial adapter. In my case, it's a grounding problem.

 

[Craig Curtin]

Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...

So have you reset this switch and followed the OP to set it up - or is this one you have inherited with unknown settings ?

Try the base firmware reload from scratch - assuming a restart gives you something to work with and allows you into the initial boot screen.

Have you looked at flow control settings and is this a known console cable you are using - or just a spare cable lying around ?

Craig

 

[jkieffer72]

So have you reset this switch and followed the OP to set it up - or is this one you have inherited with unknown settings ?

Try the base firmware reload from scratch - assuming a restart gives you something to work with and allows you into the initial boot screen.

Have you looked at flow control settings and is this a known console cable you are using - or just a spare cable lying around ?

Craig

New to me switch from eBay. Looks hardly used, but does not appear to be at default configuration.

I am trying to follow the guide to reset as configuration (password) is unknown. I have not been able to access the serial console yet.

The management (network) port pulls an IP Address from DHCP. I can reach the web interface, but again password is unknown and not any of the defaults I have found referenced online.

Flow control is set to none. The console cable was new in a package of accessories that came with the switch. (Power cords, rack hardware, stacking cables)

Going to go try again a couple of different ways...

 

[Craig Curtin]

New to me switch from eBay. Looks hardly used, but does not appear to be at default configuration.

I am trying to follow the guide to reset as configuration (password) is unknown. I have not been able to access the serial console yet.

The management (network) port pulls an IP Address from DHCP. I can reach the web interface, but again password is unknown and not any of the defaults I have found referenced online.

Flow control is set to none. The console cable was new in a package of accessories that came with the switch. (Power cords, rack hardware, stacking cables)

Going to go try again a couple of different ways...

OK well we would expect the serial cable to work - just confirming is it a DB9 serial or a USB Serial ?

Forgive me for asking - but you are plugging it into the correct (serial/console) management port - not the LAN management port (as they are both RJ45)

DO you have any other apps on your machine that would be accessing the serial port ?

If it is a USB cable then when you plug it into your machine you should hear the usual windows tones for a newly found device and the toast icon should popup saying it has been found

You can then go into device manager and look for the newly created serial port.

Craig

 

[jkieffer72]

OK well we would expect the serial cable to work - just confirming is it a DB9 serial or a USB Serial ?

Forgive me for asking - but you are plugging it into the correct (serial/console) management port - not the LAN management port (as they are both RJ45)

DO you have any other apps on your machine that would be accessing the serial port ?

If it is a USB cable then when you plug it into your machine you should hear the usual windows tones for a newly found device and the toast icon should popup saying it has been found

You can then go into device manager and look for the newly created serial port.

Craig

Thanks for the input.

I think it is something with my USB to Serial adapter.

I just tried from another computer that has a serial port and was able to connect without issue.

Edit/Update- basic configuration is done now. I have access via ssh and http. Now to see if I can get the rest of the network connected.. and figure out how to quiet it down further...

 

[Craig Curtin]

Thanks for the input.

I think it is something with my USB to Serial adapter.

I just tried from another computer that has a serial port and was able to connect without issue.

Edit/Update- basic configuration is done now. I have access via ssh and http. Now to see if I can get the rest of the network connected.. and figure out how to quiet it down further...

Good one - thanks for letting us know

Craig