I do not believe there are any known issues with those NICs.
I have just setup my Mellanox CX354 (single port) with a 40GB QSFP cable straight into 1/2/1 and 1/2/6 and had no problems.
Craig
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I have just setup my Mellanox CX354 (single port) with a 40GB QSFP cable straight into 1/2/1 and 1/2/6 and had no problems.
Craig
There is some physical weirdness going on inside of this switch. I opened it up and took out the 40g board. There was a piece of something down in the pins of one of the headers on the motherboard that are used to connect to the 40g board. So someone was definitely in there before tinkering, maybe trying to solve the same problem I am having. I blew it out with canned air, as well as the other header and the opposing connectors on the 40g board, but it made no difference. It just stinks I got the last one from the seller, or I would have just opted for a replacement. Are the "XL" LEDs on the front supposed to light up?
TLDR - Yes.
Longer read - I'd try and get through the first 40 pages of posts in this thread.
This is still a very active thread - start at the end and go backwards - you'll see. Fewer posts because people find the answers in this thread. Thanks, I had seen it was still active and was not sure where to start or what to search for. I'll follow your advice!
TLDR - Yes.
Longer read - I'd try and get through the first 40 pages of posts in this thread.
Just wanted to ask - can the 4 ports on the back of the 6610 all be used as 40GB for NIC attachment - or do the breakout ports have to be used as breakout ports
I still have not managed to solve my problem with the disabling of ports in ESXi when i make VLAN changes - but by using 40GB NICs i have two servers up and running directly attached to the QSFP stacking ports in the rear - in my home lab - would be nice if i could get my last host connected at 40GB by removing the fanout out cable and direct attaching with a QSFP nic
Craig
I still have not managed to solve my problem with the disabling of ports in ESXi when i make VLAN changes - but by using 40GB NICs i have two servers up and running directly attached to the QSFP stacking ports in the rear - in my home lab - would be nice if i could get my last host connected at 40GB by removing the fanout out cable and direct attaching with a QSFP nic
Craig
Pretty sure the two breakout ports are hardcoded to be so on the 6610.
6450 - which do i believe ?
Guys - 2 x 6450 - licensed and configured as per OP document - no issues
I have a couple of different SFP+ cables
1) 10GTek - AOC SFP+ 20m fibre - as per here - https://www.amazon.com.au/dp/B08TSZ5TH1?ref=ppx_yo2ov_dt_b_product_details&th=1
2) No name 1m DAC SFP+
3) Dell 5m DAC SFP+ - known to work between 2 x 6610 and a 6610 and an Intel 10GB card
When i link the two 6450 through their SFP+ ports i get this
So does it look like the Intel coded cable is working at 10GB ? (i will test shortly with a client card to the switch)
And should i be concerned about the not supported statement after both of the DACs ?
Craig
Guys - 2 x 6450 - licensed and configured as per OP document - no issues
I have a couple of different SFP+ cables
1) 10GTek - AOC SFP+ 20m fibre - as per here - https://www.amazon.com.au/dp/B08TSZ5TH1?ref=ppx_yo2ov_dt_b_product_details&th=1
2) No name 1m DAC SFP+
3) Dell 5m DAC SFP+ - known to work between 2 x 6610 and a 6610 and an Intel 10GB card
When i link the two 6450 through their SFP+ ports i get this
Code:
SSH@afsydney-6450-Bottom>show int e 1/2/3
10GigabitEthernet1/2/3 is up, line protocol is up
Port up for 4 minute(s) 5 second(s)
Hardware is 10GigabitEthernet, address is cc4e.24a8.4540 (bia cc4e.24a8.4573)
Configured speed 10Gbit, actual 10Gbit, configured duplex fdx, actual fdx
Member of L2 VLAN ID 1, port is untagged, port state is FORWARDING
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
Link Error Dampening is Disabled
STP configured to ON, priority is level0, mac-learning is enabled
Flow Control is enabled
Mirror disabled, Monitor disabled
Mac-notification is disabled
Not member of any active trunks
Not member of any configured trunks
No port name
MTU 1500 bytes, encapsulation ethernet
300 second input rate: 344 bits/sec, 0 packets/sec, 0.00% utilization
300 second output rate: 14528 bits/sec, 11 packets/sec, 0.00% utilization
904 packets input, 97366 bytes, 0 no buffer
Received 10 broadcasts, 0 multicasts, 894 unicasts
0 input errors, 0 CRC, 0 frame, 0 ignored
0 runts, 0 giants
7803 packets output, 1149522 bytes, 0 underruns
Transmitted 2831 broadcasts, 4135 multicasts, 837 unicasts
0 output errors, 0 collisions
Relay Agent Information option: Disabled
Egress queues:
Queue counters Queued packets Dropped Packets
0 0 0
1 0 0
2 0 0
3 0 0
4 0 0
5 0 0
6 0 0
7 0 0
SSH@afsydney-6450-Bottom>
So it thinks the port is running at 10G and i get the same report on the other end - however a show media does this
SSH@afsydney-6450-Bottom>show media e 1/2/3
Port 1/2/3: Type : 1GE M-SX(SFP)
Vendor: Intel Corp Version: A
Part# : FTLX8571D3BCV-IT Serial#: INA10M20058
and when i switch to the Copper DAC i get this (yet it is passing traffic between switches so seems to work ?)
SSH@afsydney-6450-Bottom>show media e 1/2/3
Port 1/2/3: Type : 10GE Passive Twinax 1m (SFP +) (Not supported)
Vendor: FCI Electronics Version: G
Part# : 10110818-2010LF Serial#: CN1913SA102L021
SSH@afsydney-6450-Bottom>
And this is with a Dell 10GB DAC 5m cable that i know works between two of my 6610 switches and also from an Intel client to a 6610 at 10GB
SSH@afsydney-6450-Bottom>show media e 1/2/3
Port 1/2/3: Type : 10GE Passive Twinax 5m (SFP +) (Not supported)
Vendor: Molex Inc. Version:
Part# : 74752-9507 Serial#: 012730026And should i be concerned about the not supported statement after both of the DACs ?
Craig
Just as a follow up to this - i have used the same Intel Coded fibre cable now between a 6610 1/3/x port and an Intel Dual port 82599 card in a linux box and it will not come up at 10GB either
On the 6610 it shows
Type ? for a list
SSH@6610-Basement#show media e 1/3/5
Port 1/3/5: Type : 1GE M-SX(SFP)
Vendor: Intel Corp Version: A
Part# : FTLX8571D3BCV-IT Serial#: INA10M20058
SSH@6610-Basement#
So it looks like there is a problem with this cable and how it is coded
I thought the Brocade were meant to be pretty agnostic in terms of cables in the SFP+ ports ?
Craig
On the 6610 it shows
Type ? for a list
SSH@6610-Basement#show media e 1/3/5
Port 1/3/5: Type : 1GE M-SX(SFP)
Vendor: Intel Corp Version: A
Part# : FTLX8571D3BCV-IT Serial#: INA10M20058
SSH@6610-Basement#
So it looks like there is a problem with this cable and how it is coded
I thought the Brocade were meant to be pretty agnostic in terms of cables in the SFP+ ports ?
Craig
I thought i might try and add to the knowledgepool so tried fodeeshas debug using the serial console - linked here
But i got an error on i2c read - when doing this on a 6450 - is this not a supported command ? Does it only work with 6610 ?
@fohdeesha - if i wanted to check the eeprom in an SFP+ Module in (say 1/3/1 on a 6610) - can you advise the eeprom address for that ?
Reading over the post of yours that i quoted above i can see that this was for a QSFP+ module so there is no way that address would work on a 6450 - but still interesting to know if this debug console works in the same way ? I did manage to enter the console but the commands appear to be different between the two environments.
I have read the Siemon document that you linked to so i think i know what i am looking for - i do intend to send this cable back for a refund but am happy to do some exploring for the greater good (with a bit of direction)
Craig
Last edited:
So, the XL710 seems to be incompatible with [at least] the ICX6610-24p. No amount of tinkering with settings of the card itself could get it to work. No matter the OS...nothing. The only way it would work is to change the port config to 4x10 and have four 10 gigabit connections. I figured I'd spent enough on this already (2 spare DACs, another XL710), so what's another $60? So I ordered a Mellanox CX353A-FCBT. No issues, whatsoever.
EDIT: Don't discount it totally. Maybe it doesn't work with DACs, but may work with optics. Who knows. I only needed 1.5m worth, so I found no point of getting an AOC cable, or didn't need the distance to buy 2 optics modules and a fiber patch cable.
EDIT: Don't discount it totally. Maybe it doesn't work with DACs, but may work with optics. Who knows. I only needed 1.5m worth, so I found no point of getting an AOC cable, or didn't need the distance to buy 2 optics modules and a fiber patch cable.
The only ICX devices which support Multi-Gig are the -ZP variants. Some people have had success using Multi-Gig SFP+ modules that handle the speed shift themselves, there are comments in this thread about that (they are Aquantia devices IIRC).
I've got a broken ICX7450-48 here:
Any ideas what's broken? SPI Flash fried? Repairable?
Code:
PLATFORM SP
Config partition mounted.
eUSB flash drive is ready No External USB Drive(s) Mounted . Please download UFI image to install packages
/etc/process_packages.sh: line 148: can't create /.pkg/error.log: nonexistent directory
Package installation is incomplete, related functionalities may not be available
dhcpd executable file is not present.
Starting up system statistics daemon...
Starting systatsd done
Starting up health monitor daemon...
Log Level 7: Daemonized systatsd
Systatsd Logging Env is not set, initializing loglevel to: 7
Setting connection to systemlogger
Starting hmond done
Hmond Logging Env is not set, initializing loglelvel to: I
Setting connection to systemlogger
Enabling core#1
core#1 enabled
Kernel crashdump is disabled.
Resouces folder created successfully!
dma_mem_base: 0x96000000, dma_mem_len: 0x8000000, warm_mem_base: 0xdf500000, warm_mem_len: 0xb00000
Creating TUN device
Enabling time stamp.
*** Crash dump information is available, please collect it using "supportsave" CLI ***
Starting the FastIron.
USER=root
boot_partition=pri_partition
HOME=/
GCOV_PREFIX=/fast_iron
ethaddr=d4:c1:9e:00:00:7f?
TERM=vt102
PATH=/sbin:/usr/sbin:/bin:/usr/bin
ip=192.168.60.37:192.168.60.1:192.168.0.1:255.255.255.0:SP:eth0:off
SHELL=/bin/sh
PWD=/
[sp-gpio]:sil_gpiochip_register: gpiochip registeration
i2c-0 smbus iproc-smb-1 SMBus adapter 3
i2c-1 smbus iproc-smb-1 SMBus adapter 3
FIPS Disabled:PORT NOT DISABLED
platform type 72
OS>
Loaded Image SPR08090mc from Pri Built on (UTC): Mon Oct 4 17:51:11 2021
Uboot and FI booted from different partitions, Uboot: Secondary, FI: Primary
Cannot auto correct cross boot combination, other partition of uboot does not have recommended uboot for current booted FI image
Enabling Console Logging
sil_sp_spi_write: Write failed for dev 10, err: Connection timed out
Failed to write env parameters
Error: can't write sil_env to flash
sil_sp_spi_write: Write failed for dev 10, err: Connection timed out
Failed to write env parameters
Error: can't write sil_env to flash
Applying factory defaults..
INFO: empty config data in the primary area, try to read from backup
INFO: empty config data in the backup area also
INFO: startup config data is not available, try to read from backup
INFO: startup config data in the backup area is not available
Hotplugger Daemon Initialized ..
set_board_level: gi_board_type = 100[ 146.768001] linux-kernel-bde: map phys range 0x48000000-0x48040000 to 0xb0b2f000-0xb0b6f000
[ 146.870302] linux-kernel-bde: map phys range 0x42000000-0x42040000 to 0xb0aef000-0xb0b2f000
[ 146.970704] linux-kernel-bde: map phys range 0x42040000-0x42080000 to 0xb0aaf000-0xb0aef000
[ 147.225232] linux-kernel-bde: map phys range 0x88700000-0x90700000 to 0xa8aaf000-0xb0aaf000
DMA pool size: 134217728
SOC unit 0 attached to PCI device BCM56548_A0
0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
u=0 p=50 Using SDK PHY driver for BCM84756/7/9
u=0 p=51 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
SOC unit 1 attached to PCI device BCM56548_A0
1:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
u=1 p=42 Using SDK PHY driver for BCM84756/7/9
u=1 p=43 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
[ 223.470012] [BrcdSoftlockup]: BUG: soft lockup - CPU#0 stuck for 31s! [bcmCNTR.0:1559]
[ 223.565004] [BrcdSoftlockup]: core collection to be started now @ core(1)...
[ 223.649504] [BrcdSoftlockup]: WD_TOUT: Signalling pid: 1226, process: SigHdlrTsk, tgid: 6 to initiate core collection
[ 223.776708] [BrcdSoftlockup]: Switch will be reset after core collection
Watchdog fired... Killing FastIron
TS: 186717968, Watchdog counter 31
watchdog_skip_count:1, elapsed_ts_since_last_kick:124195115, last_skipped_kick_ts:95217194034, last_watchdog_kick_ts:95092998919
Application received signal -> SIGNUM#6
[ 228.850435] [BrcdSoftlockup]: (5) secs expired for core-collection . Now time to dump FI thread backtraces to warm memory
[ 228.982835] [BrcdSoftlockup]:
[ 228.982835] content dumping finished..
Tuning CFS scheduler parameters...
Compressing Console file
Copying fitrace errorlog file to flash
CORE_PATTERN:PID=1224 !FastIron UID=0 GID=0 sig=6
Sun Dec 26 05:25:17 UTC 2004: Dumping core file to /tmp.gz, this will take couple of minutes ...
[ 313.650442] [BrcdSoftlockup]: Time elapsed since core-collection started (292) secs. Max Timeout(1200) secs
[ 403.650460] [BrcdSoftlockup]: Time elapsed since core-collection started (375) secs. Max Timeout(1200) secs
Aborted (core dumped)
Sun Dec 26 05:28:19 UTC 2004: Core file collected as /tmp.gz, processing the core file ...
Sun Dec 26 05:28:19 UTC 2004: Removing the oldest core file:core_929-28620.gz of size 2832 kbytes
seq 28621
Sun Dec 26 05:28:19 UTC 2004: Calculating checksum: core_1224_FastIron-28622_2004-12-26_05-28-19.gz
Sun Dec 26 05:28:19 UTC 2004: Checking integrity of core_1224_FastIron-28622_2004-12-26_05-28-19.gz
...Have you tried reflashing with 8090 and matching boot loader from the uBoot environment?
Failing that, you could try a NAND scrub as a last resort, then flash again.
That one SPI error is concerning.
Failing that, you could try a NAND scrub as a last resort, then flash again.
That one SPI error is concerning.
Yes, that is the first thing I did after receiving the switch.
Yes, and I believe it is the root cause for the CPU lockup.
Because
saveenv in u-boot also didn't work (aka didn't save anything) and took way longer to execute / return than it shouldYeah then it's probably toast. Only reason I mentioned reflash first was the cross-boot warning which means the uBoot version and the flash version are mismatched somewhere, or you flashed an UFI image from uBoot so the boot code didn't properly update.
Maybe recoverable via JTAG if you have the right stuff? That's an area where have no experience though.
I always get that warning when following @fohdeesha 's guide, i.e. first flashing the new 10118 u-boot, then the mc image and from there the UFI image
As the 7XXX series has redundant SPI flash, I might be able to revive it by selecting the other SPI flash?
I looked through the source and there should be a
switch_uboot command...Just got a ICX6450-48P that powers on but I can't get anything from either the console port or the management port.
I've used the same RS232 USB RJ45 that works on my ICX7150-C12P
Is there any possibility that the console port or the management port gets disabled? If so how do I reset it or gain access to reconfigure it? Also is there a default IP for the management port?
Thanks!
I've used the same RS232 USB RJ45 that works on my ICX7150-C12P
Is there any possibility that the console port or the management port gets disabled? If so how do I reset it or gain access to reconfigure it? Also is there a default IP for the management port?
Thanks!