Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
LodeRunner said:
As long as both sides of the LAG are the same mode, you shouldn't have a problem. Like vangoose, I have an ICX connected to an Arista using LACP; no issues.
Click to expand...
I guess an extension of this question - if i have 2 x 6450 - do i just connect them together through the front SFP+ ports with a DAC and all done ?

Craig
 
L

LodeRunner

Active Member
Apr 27, 2019
554
236
43
Craig Curtin said:
I guess an extension of this question - if i have 2 x 6450 - do i just connect them together through the front SFP+ ports with a DAC and all done ?

Craig
Click to expand...
You should configure the ports first, otherwise you'd have spanning tree will down a port or you forgot about spanning tree and you have a loop.

Or down the ports, cable them, configure, then enable them.

I don't know if the 6450 has any automatic stacking; the 7450's I deployed at work automatically stacked using the 40G ports, but I connected them in a linear stack first, waited for it to configure, then closed the loop.

Here're some config snippets from my 7450 at home using both 40G ports as a LACP bundle to my Arista core:
Code: 
lag 40G-Core dynamic id 4
ports ethe 1/3/1 ethe 1/4/1
port-name 40G-Core:E35 ethernet 1/3/1
port-name 40G-Core:E36 ethernet 1/4/1
!
...
interface lag 4
spanning-tree 802-1w admin-pt2pt-mac
!
VLANs are assigned to it the same as any other port, referencing the LAG port (in this case lg4 for me) rather than the physical ports.

Here's the Arista side, which is more or less Cisco syntax:
Code: 
interface Port-Channel1
   description 1G Core LAG
   switchport trunk allowed vlan <range>
   switchport mode trunk
   port-channel min-links 1
!
...
interface Ethernet35
   description To Core:1/3/1
   switchport mode trunk
   channel-group 1 mode active
!
interface Ethernet36
   description To Core:1/4/1
   switchport mode trunk
   channel-group 1 mode active
!
 
Last edited:
  • Like
Reactions: thebwack
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
LodeRunner said:
You should configure the ports first, otherwise you'd have spanning tree will down a port or you forgot about spanning tree and you have a loop.

Or down the ports, cable them, configure, then enable them.

I don't know if the 6450 has any automatic stacking; the 7450's I deployed at work automatically stacked using the 40G ports, but I connected them in a linear stack first, waited for it to configure, then closed the loop.

Here're some config snippets from my 7450 at home using both 40G ports as a LACP bundle to my Arista core:
Code: 
lag 40G-Core dynamic id 4
ports ethe 1/3/1 ethe 1/4/1
port-name 40G-Core:E35 ethernet 1/3/1
port-name 40G-Core:E36 ethernet 1/4/1
!
...
interface lag 4
spanning-tree 802-1w admin-pt2pt-mac
!
VLANs are assigned to it the same as any other port, referencing the LAG port (in this case lg4 for me) rather than the physical ports.

Here's the Arista side, which is more or less Cisco syntax:
Code: 
interface Port-Channel1
   description 1G Core LAG
   switchport trunk allowed vlan <range>
   switchport mode trunk
   port-channel min-links 1
!
...
interface Ethernet35
   description To Core:1/3/1
   switchport mode trunk
   channel-group 1 mode active
!
interface Ethernet36
   description To Core:1/4/1
   switchport mode trunk
   channel-group 1 mode active
!
Click to expand...

Cheers mate - much appreciated.

No stacking ports per se on the 6450 - i think they are really designed to be closet switches on the edge of the network - cabled back through 10GB to a central 6610 etc

Craig
 
L

LodeRunner

Active Member
Apr 27, 2019
554
236
43
J

jkieffer72

New Member
Dec 7, 2022
5
2
3
Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...
 
J

jkieffer72

New Member
Dec 7, 2022
5
2
3
jkieffer72 said:
Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...
Click to expand...
I was reading thru this thread and saw a few suggestions:

Reboot windows. And/or uninstall/reinstall the com port drivers.
Try a real serial port, not a USB converter.
 
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
jkieffer72 said:
Hoping someone has an answer that will help...

I just received my 6610 switch today and am trying to access it via the (serial) console port.

I tried the default 96008N1 using Putty and Terra Term.

Characters display, but they are non-sensical. i.e., ööïïïïïï¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¤®ï¶¤®ïa¤®ïïfa¤®ïïïËV6i½ïïï¿ï#s¿ïïïïïû+6½ïïïï¿ï¿#s¿ïïïïï¦+6½ïïï{#s¿ïïïï¶a¤®ïï®a¤®ïïï*6½ïïïooov*lR.ooooooo}*lR.ooooooo&ooo¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz¨^y½y[^jÙYöVmvöËÔéz6$6ïï{ïï¿ï

àààààààààààààà

I have tried a variety of baud rates thinking perhaps the default 9600 had been changed. I get different characters, but still nothing recognizable.

Any suggestions? I have tried several usb to serial adapters I have from Arduino projects. The PC I am using does not have a built in serial port...

I will try again later, off to a family dinner... I have another PC that I think has a serial port and can give that a shot...
Click to expand...
So have you reset this switch and followed the OP to set it up - or is this one you have inherited with unknown settings ?

Try the base firmware reload from scratch - assuming a restart gives you something to work with and allows you into the initial boot screen.

Have you looked at flow control settings and is this a known console cable you are using - or just a spare cable lying around ?

Craig
 
J

jkieffer72

New Member
Dec 7, 2022
5
2
3
Craig Curtin said:
So have you reset this switch and followed the OP to set it up - or is this one you have inherited with unknown settings ?

Try the base firmware reload from scratch - assuming a restart gives you something to work with and allows you into the initial boot screen.

Have you looked at flow control settings and is this a known console cable you are using - or just a spare cable lying around ?

Craig
Click to expand...
New to me switch from eBay. Looks hardly used, but does not appear to be at default configuration.

I am trying to follow the guide to reset as configuration (password) is unknown. I have not been able to access the serial console yet.

The management (network) port pulls an IP Address from DHCP. I can reach the web interface, but again password is unknown and not any of the defaults I have found referenced online.

Flow control is set to none. The console cable was new in a package of accessories that came with the switch. (Power cords, rack hardware, stacking cables)

Going to go try again a couple of different ways...
 
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
jkieffer72 said:
New to me switch from eBay. Looks hardly used, but does not appear to be at default configuration.

I am trying to follow the guide to reset as configuration (password) is unknown. I have not been able to access the serial console yet.

The management (network) port pulls an IP Address from DHCP. I can reach the web interface, but again password is unknown and not any of the defaults I have found referenced online.

Flow control is set to none. The console cable was new in a package of accessories that came with the switch. (Power cords, rack hardware, stacking cables)

Going to go try again a couple of different ways...
Click to expand...
OK well we would expect the serial cable to work - just confirming is it a DB9 serial or a USB Serial ?

Forgive me for asking - but you are plugging it into the correct (serial/console) management port - not the LAN management port (as they are both RJ45)

DO you have any other apps on your machine that would be accessing the serial port ?

If it is a USB cable then when you plug it into your machine you should hear the usual windows tones for a newly found device and the toast icon should popup saying it has been found

You can then go into device manager and look for the newly created serial port.

Craig
 
J

jkieffer72

New Member
Dec 7, 2022
5
2
3
Craig Curtin said:
OK well we would expect the serial cable to work - just confirming is it a DB9 serial or a USB Serial ?

Forgive me for asking - but you are plugging it into the correct (serial/console) management port - not the LAN management port (as they are both RJ45)

DO you have any other apps on your machine that would be accessing the serial port ?

If it is a USB cable then when you plug it into your machine you should hear the usual windows tones for a newly found device and the toast icon should popup saying it has been found

You can then go into device manager and look for the newly created serial port.

Craig
Click to expand...
Thanks for the input.

I think it is something with my USB to Serial adapter.

I just tried from another computer that has a serial port and was able to connect without issue.

Edit/Update- basic configuration is done now. I have access via ssh and http. Now to see if I can get the rest of the network connected.. and figure out how to quiet it down further...
 
Last edited:
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
jkieffer72 said:
Thanks for the input.

I think it is something with my USB to Serial adapter.

I just tried from another computer that has a serial port and was able to connect without issue.

Edit/Update- basic configuration is done now. I have access via ssh and http. Now to see if I can get the rest of the network connected.. and figure out how to quiet it down further...
Click to expand...
Good one - thanks for letting us know

Craig
 
R

ramicio

Member
Nov 30, 2022
78
14
8
Okay, here's what I've been through hardware-wise

  1. New DAC cables (two), one end coded for Brocade, the other end for Intel.
  2. Another XL710 NIC
The behavior has not changed. I cannot use DHCP, and I can only establish a link via static settings. When I do form a static connection, these are the errors I'm getting on the interface in Ubuntu:

Code: 
eth2: flags=4163(UP,BROADCAST,RUNNING,MULTICAST)  mtu 1500
    inet 192.168.1.43  netmask 255.255.255.0   broadcast 192.168.1.255
    inet6 fe80::3efd:feff:feb2:cd30  prefixlen 64  scopeid 0x20(link)
    ether 3c:fd:fe:b2:cd:30  txqueuelen 1000  (Ethernet)
    RX packets 741  bytes 46100 (46.1 KB)
    RX errors 382  dropped 197  overruns 0  frame 382
    TX packets 39  bytes 6887 (6.8 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Pinging is fine to and from, and even with larger byte sizes like 1,400 bytes.

On the switch there are zero RX or TX errors:

Code: 
40GigabitEthernet1/2/1 is up, line protocol is up
  Port up for 16 minute(s) 37 second(s)
  Hardware is 40GigabitEthernet, address is 748e.f8fe.93f6 (bia 748e.f8fe.940f)
  Interface type is 40Gig Fiber
  Configured speed 40Gbit, actual 40Gbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual none
  Member of L2 VLAN ID 1, port is untagged, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is enabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  MTU 1500 bytes, encapsulation ethernet
  300 second input rate: 48 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 2168 bits/sec, 3 packets/sec, 0.00% utilization
  56352 packets input, 6296114 bytes, 0 no buffer
  Received 503 broadcasts, 7331 multicasts, 48518 unicasts
  0 input errors, 0 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  350121 packets output, 73835998 bytes, 0 underruns
  Transmitted 143184 broadcasts, 121844 multicasts, 85093 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled

Egress queues:
Queue counters    Queued packets    Dropped Packets
    0              347387                   0
    1                   0                   0
    2                   0                   0
    3                   0                   0
    4                   0                   0
    5                  19                   0
    6                   0                   0
    7                2719                   0

Tried disabling LLDP for that port (a suggestion), made things worse, re-enabled. Port 1/2/6 acts the same.

Should I be looking for a new switch (bad ports), or different type of NIC?

EDIT:
If I ping with 1,450 bytes I get 0% packet loss. If I ping with 1,451 bytes or more, I get 100% packet loss.
 
Last edited:
C

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
ramicio said:
Okay, here's what I've been through hardware-wise

  1. New DAC cables (two), one end coded for Brocade, the other end for Intel.
  2. Another XL710 NIC
The behavior has not changed. I cannot use DHCP, and I can only establish a link via static settings. When I do form a static connection, these are the errors I'm getting on the interface in Ubuntu:

Code: 
eth2: flags=4163(UP,BROADCAST,RUNNING,MULTICAST)  mtu 1500
    inet 192.168.1.43  netmask 255.255.255.0   broadcast 192.168.1.255
    inet6 fe80::3efd:feff:feb2:cd30  prefixlen 64  scopeid 0x20(link)
    ether 3c:fd:fe:b2:cd:30  txqueuelen 1000  (Ethernet)
    RX packets 741  bytes 46100 (46.1 KB)
    RX errors 382  dropped 197  overruns 0  frame 382
    TX packets 39  bytes 6887 (6.8 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Pinging is fine to and from, and even with larger byte sizes like 1,400 bytes.

On the switch there are zero RX or TX errors:

Code: 
40GigabitEthernet1/2/1 is up, line protocol is up
  Port up for 16 minute(s) 37 second(s)
  Hardware is 40GigabitEthernet, address is 748e.f8fe.93f6 (bia 748e.f8fe.940f)
  Interface type is 40Gig Fiber
  Configured speed 40Gbit, actual 40Gbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual none
  Member of L2 VLAN ID 1, port is untagged, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is enabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  MTU 1500 bytes, encapsulation ethernet
  300 second input rate: 48 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 2168 bits/sec, 3 packets/sec, 0.00% utilization
  56352 packets input, 6296114 bytes, 0 no buffer
  Received 503 broadcasts, 7331 multicasts, 48518 unicasts
  0 input errors, 0 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  350121 packets output, 73835998 bytes, 0 underruns
  Transmitted 143184 broadcasts, 121844 multicasts, 85093 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled

Egress queues:
Queue counters    Queued packets    Dropped Packets
    0              347387                   0
    1                   0                   0
    2                   0                   0
    3                   0                   0
    4                   0                   0
    5                  19                   0
    6                   0                   0
    7                2719                   0

Tried disabling LLDP for that port (a suggestion), made things worse, re-enabled. Port 1/2/6 acts the same.

Should I be looking for a new switch (bad ports), or different type of NIC?

EDIT:
If I ping with 1,450 bytes I get 0% packet loss. If I ping with 1,451 bytes or more, I get 100% packet loss.
Click to expand...
Post your switch config again - i have not used the 40G ports (but am about to when my cables arrive) - is there anything special that needs to be done to put them into 40Gb mode instead of stacking mode ?

Craig
 
R

ramicio

Member
Nov 30, 2022
78
14
8
Craig Curtin said:
Post your switch config again - i have not used the 40G ports (but am about to when my cables arrive) - is there anything special that needs to be done to put them into 40Gb mode instead of stacking mode ?

Craig
Click to expand...
Code: 
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-24p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
 router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname switcheroo
ip dhcp-client disable
!
no telnet server
username root password .....
!
!
!
!
!
!
!
!
!
interface ethernet 1/1/3
 inline power
!
interface ethernet 1/1/6
 inline power
!
interface ethernet 1/1/7
 inline power
!
interface ethernet 1/3/1
 speed-duplex 10G-full
!
interface ethernet 1/3/2
 speed-duplex 10G-full
!
interface ethernet 1/3/3
 speed-duplex 10G-full
!
interface ethernet 1/3/4
 speed-duplex 10G-full
!
interface ethernet 1/3/5
 speed-duplex 10G-full
!
interface ethernet 1/3/6
 speed-duplex 10G-full
!
interface ethernet 1/3/7
 speed-duplex 10G-full
!
interface ethernet 1/3/8
 speed-duplex 10G-full
!
interface ve 1
 ip address 192.168.1.5 255.255.255.0
!
!
!
!
!
lldp run
!
!
!
!
end
 
R

ramicio

Member
Nov 30, 2022
78
14
8
So, I used the Ethernet Port Config utility to set the port from 1x40g to 4x10g. It gives me 4 interfaces on the server. On the switch I moved the DAC over to the 1/2/2 - 1/2/5 breakout port. Gave /etc/network/interfaces 4 entries, used DHCP. Each interface pulled an address right away and each interface has ZERO errors. Are these cards not compatible with this switch when it comes to full 40g? Or can 1/2/1 and 1/2/6 both be dead whilst the 2 breakout ports are fine?
 
  • Like
Reactions: itronin
R

ramicio

Member
Nov 30, 2022
78
14
8
I've also noticed that none of the "XL" LEDs on the front ever come on. Not during boot, and not even when a link is established (albeit with RX errors on the server). XL2-5 should at least be lit, because that is a successful link without any errors on the counters.
 
R

ramicio

Member
Nov 30, 2022
78
14
8
I got a partial refund for the switch. Is it worth it buying another one? Is there a known issue with these particular NICs?
 
You must log in or register to reply here.
Top Bottom