Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

itronin

Well-Known Member
Nov 24, 2018
1,233
793
113
Denver, Colorado
special things made public when we hit page 200
Doing my part to extend the thread count.

So let's play the guessing game:

* Did the guinea pigs running the wheel on the eternal license making forge die? Or did you develop a fondness for Peruvian cuisine

* You've transferred the guinea pigs and eternal license making forge to @infoMatt

* Arris/Ruckus/Whatever the @#$@#$ company owns the hard and softs has decided to unlock the licenses for everyone in the next firmware release

* In addition to the last one there is now a plex / emby plug-in for the linux supervisor that handles 10+ concurrent 4k transcodes with no switching routing performance reduction

* This thread is actually the cause of the STH forum server failure and will automatically be deleted when the thread count hits 200 pages and all information contained in the thread will be forever lost to the sands of time
 

Spearfoot

Active Member
Apr 22, 2015
111
51
28
I can help us hit 200. :D

Still curious which DAC breakout cable is favored. To make this post more on-topic. ;)
I would bet upwards of $10 that these Brocade/Ruckus switches will work with just about any breakout cables. :)

I bought a pair of these Arista cables on eBay and they work fine. They're sold out now, but there are similar cables available.

 

Spearfoot

Active Member
Apr 22, 2015
111
51
28
Looks like the cheapest ones out there today are Dell Force10. Any reason those wouldn't work?
Can't see any reason why they wouldn't. I remember seeing those, too, but they're 5M in length -- way too long for my needs. And if for some reason they don't work you can return 'em.
 

hmw

Active Member
Apr 29, 2019
570
226
43
I was using the ICX6610 40G QSFP+ stacking port with a ConnectX-3 via a NetApp 112-00177 cable - and it worked perfectly, with the card being able to connect at 40 GbE

I replaced the ConnectX-3 with a ConnectX-4 LX - specifically a MCX4131A-GCAT - and the card just won't enable the link when connected via the NetApp cable. I then tried connecting the card via a MAM1Q00A-QSA QSFP+ to SFP adapter to one of the front SFP ports - and it worked, although it connected at 10G (as expected)

Updated drivers under ESXi - no go.

The card obviously works - I was able to get the card to do SR-IOV under ESXi - with 3 VMs having a virtual copy of the card, and it all works fine.


Anyone used the ICX6610 with a ConnectX-4 LX? Did it work ? What cables have folks used?

There's a 'Mellanox compatible' 40G QSFP+ cable from FS - Mellanox MC2210130-002 40G QSFP+ DAC Cable - but Mellanox's own MCX4131 page recommends a QSFP28 cable for the MCX4131A - surely that would NOT work with the ICX6610 stacking ports?

Any advice would be welcome - dipping my toes in the 40GbE world and it's not always smooth sailing :)

I used @fohdeesha's guide on the debug console + i2c to read the NetApp 112-00177 cable EEPROM

Code:
0000: 0c 00 02 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0010: 00 00 00 00 00 00 01 81-00 00 00 00 00 00 00 00 ................
0020: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0040: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0050: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0060: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0070: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0080: 0c 00 21 00 00 00 04 41-80 80 d5 00 00 00 00 00 ..!....A........
0090: 00 00 02 80 4d 6f 6c 65-78 20 49 6e 63 2e 20 20 ....Molex Inc.
00a0: 20 20 20 20 00 00 09 3a-31 31 32 2d 30 30 31 37     ...:112-0017
00b0: 37 20 20 20 20 20 20 20-41 30 00 00 00 00 46 90 7       A0....F.
00c0: 00 00 00 00 31 31 30 35-32 30 34 31 31 20 20 20 ....110520411
00d0: 20 20 20 20 31 31 30 34-31 35 20 20 00 00 00 0b     110415  ....
00e0: 01 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00f0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
The x0C means the cable is advertising as a QSFP and not QSFP+. That is probably why the ConnectX-4 LX card won't connect, even at 10GbE. My Mellanox branded cables should be here by the end of the week - will update the thread then
 

hmw

Active Member
Apr 29, 2019
570
226
43
In case anyone is interested in knowing which ports/devices correspond to which I2C DEVICE ID:

Code:
OS>i2c devices
Name                  ID      Addr      Bus     Type
-----------           ----    ------    ------  ------
PCA9548                0xa    0x70       0       MUX
SPD_EEPROM             0x2d   0x51       0       DEV
ONBOARD_EEPROM         0x1    0x52       0       DEV
CPLD1                  0x2e   0x23       0       DEV
CPLD2                  0x2f   0x22       0       DEV
RTC                    0x30   0x68       0       DEV
PCIE_SW                0x31   0x38       0       DEV
PSU1                   0x32   0x50       0       DEV
PSU2                   0x33   0x50       0       DEV
W83782G                0x5    0x2f       0       DEV
MARGIN_TESTER          0x34   0x18       0       DEV
PCA9535                0x35   0x20       0       DEV
PCA9535_1              0x37   0x21       0       DEV
PCA9548_1              0xb    0x76       0       DEV
PCA9535_2              0x38   0x24       0       DEV
PCA9535_3              0x39   0x26       0       DEV
24C02_POE              0x7    0x53       1       DEV
PD69000                0x6    0x14       1       DEV
PCA9546                0x2    0x70       1       MUX
PCA9535_4              0x3a   0x22       1       DEV
PCA9535_5              0x3b   0x23       1       DEV
QSFP1                  0x3f   0x50       1       DEV
QSFP2                  0x40   0x50       1       DEV
QSFP3                  0x41   0x50       1       DEV
QSFP4                  0x42   0x50       1       DEV
SFP1                   0x12   0x50       0       DEV
SFP2                   0x13   0x50       0       DEV
SFP3                   0x14   0x50       0       DEV
SFP4                   0x15   0x50       0       DEV
SFP5                   0x16   0x50       0       DEV
SFP6                   0x17   0x50       0       DEV
SFP7                   0x18   0x50       0       DEV
SFP8                   0x19   0x50       0       DEV
The four 40GbE QSFP ports on the back are 0x3F, 0x40, 0x41 and 0x42

The eight SFP ports in the front are 0x12 to 0x19
 
  • Like
Reactions: Spearfoot

Jason Antes

Active Member
Feb 28, 2020
224
76
28
Twin Cities
I still have AOC FC QSFP+ to SFP+ breakout cables from Brocade... ;)

I have to go through my stuff, I have several DAC breakouts too but they are used.

But this post if really just a bump in disguise to get this up to 200 sooner. :D
 
  • Like
Reactions: hmw

hmw

Active Member
Apr 29, 2019
570
226
43
I still have AOC FC QSFP+ to SFP+ breakout cables from Brocade... ;)

I have to go through my stuff, I have several DAC breakouts too but they are used.

But this post if really just a bump in disguise to get this up to 200 sooner. :D
We're on page #199 now ... 18 more posts to go

c'mon people ... post !!
 
  • Like
Reactions: Jason Antes

EngineerNate

Member
Jun 3, 2017
68
16
8
35
I ordered a Dell Force10 DAC breakout to try. If it doesn't work on my 7450 I'll send it to someone with a 6610 free to verify compatibility for future reference.