Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

umass1966

New Member
Nov 2, 2019
10
8
3
Can anybody help me with this question. i have icx7150-c12p switch and want to setup a 10gbe network like freenas server - switch - computer. want suggestions for network card , optic sfp+ and fiber cable . thanks.
 

French Chamallow

New Member
Mar 15, 2019
17
3
3
did not read this message unless you want to laugh ....
I forgot to add the firewall rules in my LANTEST :)


still sorry to annoy you but I assure you that I look before posting!

I did some tests tonight and My icx does not ping the ip 192.168.2.1
my pfsense ping well the ip of ve 100 in 192.168.2.2
I see a continuous traffic on the LANTEST interface ( 120B)

I added a transit vlan 100
I added the port 1/1/24
gave it to him ip 192.168.2.2

I went in my pfsense and I added a LANTEST on one of the free physical ports of my pfsense in 192.168.2.1

I added a gateway in System / Routing / Gateways
name ICX6450
LANTEST interface
Gateway 192.168.2.2


I specify that I left all my config of origin and thus my main lan works on another physical inerface of my pfsense ( 192.168.0.1)

SSH @ ICX6450 (config) #show ip route
Total number of IP routes: 3
Type Codes - B: BGP D: Connected O: OSPF R: RIP S: Static; Cost - Dist / Metric
BGP Codes - i: iBGP e: eBGP
OSPF Codes - i: Inter Area 1: External Type 1 2: External Type 2
Destination Gateway Port Cost Type Uptime
1 0.0.0.0/0 192.168.0.1 ve 1 1/1 S 4d3h
2 192.168.0.0/24 DIRECT ve 1 0/0 D 4d3h
3 192.168.2.0/24 DIRECT ve 100 0/0 D 1h14m


information on my 1/1/24
GigabitEthernet1/1/24 is up, line protocol is up
Port up for 1 hour(s) 38 minute(s) 28 second(s)
Hardware is GigabitEthernet, address is redacted (bia redacted)
Configured speed auto, actual 1Gbit, configured duplex fdx, actual fdx
Configured mdi mode AUTO, actual MDIX
Member of L2 VLAN ID 100, port is untagged, port state is FORWARDING
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
Link Error Dampening is Disabled
STP configured to ON, priority is level0, mac-learning is enabled
Flow Control is config enabled, oper enabled, negotiation disabled
Mirror disabled, Monitor disabled
Mac-notification is disabled
Not member of any active trunks
Not member of any configured trunks
No port name
Inter-Packet Gap (IPG) is 96 bit times
MTU 1500 bytes, encapsulation ethernet
300 second input rate: 1000 bits/sec, 1 packets/sec, 0.00% utilization
300 second output rate: 1008 bits/sec, 1 packets/sec, 0.00% utilization
7470 packets input, 478194 bytes, 0 no buffer
Received 1 broadcasts, 0 multicasts, 7469 unicasts
0 input errors, 0 CRC, 0 frame, 0 ignored
0 runts, 0 giants
7489 packets output, 479410 bytes, 0 underruns
Transmitted 22 broadcasts, 0 multicasts, 7467 unicasts
0 output errors, 0 collisions
Relay Agent Information option: Disabled


my ping

SSH@ICX6450#ping 192.168.0.1
Sending 1, 16-byte ICMP Echo to 192.168.0.1, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 192.168.0.1 : bytes=16 time=1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.
SSH@ICX6450#ping 192.168.2.1
Sending 1, 16-byte ICMP Echo to 192.168.2.1, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.
 
Last edited:

microserf

New Member
May 20, 2019
20
13
3
So I picked up an ICX 7150-C12P and updated the firmware. On boot, I'm seeing this:
Code:
Ruckus Wireless Bootloader: 10.1.14T225 (Nov 15 2018 - 04:59:18 -0800)
Booted from partition 2
DRAM:  Validate Shmoo parameters stored in flash ..... OK
ICX7150-12 (POE), PVT1
SYS CPLD VER: 0x4 Released Ver: 0xa
device 0 offset 0x0, size 0xc0000
Enter 'b' to stop at boot monitor:  0
device 0 offset 0x0, size 0xc0000
bootdelay: ===
Booting image from Primary
NAND read: device 0 offset 0x0, size 0x2000000
Skipping bad block 0x00a00000
 33554432 bytes read: OK
The second last line, "Skipping bad block 0x00a00000", concerned me so I flashed the firmware again but there was no change. The switch appears to boot fine. Comments or suggestions?
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,727
3,075
113
33
fohdeesha.com
I would reflash the bootloader as well (it's on the same SPI flash chip) but it looks like it's detected the block and marked it off which is pretty normal, you should be fine. When you need to worry is when the number of bad blocks starts rising over time, then your flash is failing
 
  • Like
Reactions: microserf

microserf

New Member
May 20, 2019
20
13
3
I would reflash the bootloader as well (it's on the same SPI flash chip) but it looks like it's detected the block and marked it off which is pretty normal, you should be fine. When you need to worry is when the number of bad blocks starts rising over time, then your flash is failing
But it's all shiny and new :(. I re-flashed the bootloader but no change. What's the mailing address for Fohdeesha's SMD Solderworks?
 

aflow

New Member
Apr 13, 2016
2
0
1
43
Hi,
Is it possible to made this modification with the ICX6450?.

Thanks.

I noticed that with the monoprice DAC breakout, it was showing up as absolutely nothing with the "show media" command. Even though it doesn't affect the behavior of the links, I figured something was up with the I2C EEPROM in the DAC cable (this is where vendor/serial/cable type info is stored for optics and DACs).

You can see the show media command can't find any vendor or cable type info for the monoprice DAC, so it shows EMPTY for the 4 breakout slots:

Code:
telnet@ICX3#show media | include 1/2/[2-5]
Port 1/2/2:  Type : EMPTY
Port 1/2/3:  Type : EMPTY
Port 1/2/4:  Type : EMPTY
Port 1/2/5:  Type : EMPTY
So I dropped down to the debug console (over serial, press ctrl+y, let go, then press m, then enter) to use some I2C read and write commands.

First, let's read the EEPROM from a proper official brocade DAC cable:

Code:
OS>i2c read 3f 0 256
0000: 0d 00 06 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0010: 00 00 00 00 00 00 01 81-00 00 00 00 00 00 00 00 ................
0020: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0040: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0050: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0060: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0070: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0080: 0d 00 23 08 00 00 00 00-00 00 00 00 67 00 00 00 ..#.........g...
0090: 00 00 01 a0 42 52 4f 43-41 44 45 20 20 20 20 20 ....BROCADE
00a0: 20 20 20 20 07 00 05 1e-35 38 2d 30 30 30 30 30     ....58-00000
00b0: 33 33 2d 30 31 20 20 20-41 20 04 05 00 00 46 08 33-01   A ....F.
00c0: 00 00 00 00 50 41 45 31-31 34 31 30 33 30 30 30 ....PAE114103000
00d0: 38 37 36 20 31 34 30 33-30 35 20 20 00 00 00 c2 876 140305  ....
00e0: 31 31 31 30 34 30 39 30-37 31 20 20 20 20 20 20 1110409071
00f0: 20 31 20 20 20 20 20 20-20 20 00 00 00 00 00 00  1        ......
You can see the first half is mostly empty, and the vendor information does not begin until the second half of the EEPROM, where you get the byte specifying the cable type, the vendor string, a serial number, etc. This is as it should be.

Now let's read the Monoprice DAC that doesn't show up properly:

Code:
OS>i2c read 41 0 256
0000: 0c 00 21 00 00 00 00 00-04 80 00 06 67 00 00 00 ..!.........g...
0010: 00 00 01 a0 4d 4f 4e 4f-50 52 49 43 45 20 04 20 ....MONOPRICE .
0020: 31 32 30 36 39 20 2e 2e-2e 2e 2e 2e 2e 2e 2e 2e 12069 ..........
0030: 77 77 77 2e 6d 6f 6e 6f-70 72 69 63 65 2e 63 6f www.monoprice.co
0040: 6d f4 31 32 30 36 39 31-34 31 30 30 30 39 36 43 m.1206914100096C
0050: 48 49 4e 41 20 20 20 00-00 00 f4 ff ff ff ff ff HINA   .........
0060: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
0070: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
0080: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
0090: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00a0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00b0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00c0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00d0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00e0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
00f0: ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................
You can see they must have been in a rush programming these, as they stuck all the vendor and serial information in the incorrect section. No wonder it shows up as empty/unknown media type! Upon closer inspection, they programmed it like an SFP+ module. This first half of the EEPROM should be populated with these values like this in an SFP+, but for a QSFP+ module, it should be in the second half.

So I used a series of i2c write commands to copy byte for byte the contents of the official brocade DAC, to the monoprice DAC, overwriting the incorrectly placed vendor info with properly placed brocade info.

The monoprice DAC EEPROM contents now looks like this:
Code:
OS>i2c read 41 0 256
0000: 0d 00 06 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0010: 00 00 00 00 00 00 01 81-00 00 00 00 00 00 00 00 ................
0020: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0040: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0050: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0060: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0070: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0080: 0d 00 23 08 00 00 00 00-00 00 00 00 67 00 00 00 ..#.........g...
0090: 00 00 01 a0 42 52 4f 43-41 44 45 20 20 20 20 20 ....BROCADE
00a0: 20 20 20 20 07 00 05 1e-35 38 2d 30 30 30 30 30     ....58-00000
00b0: 33 33 2d 30 31 20 20 20-41 20 04 05 00 00 46 08 33-01   A ....F.
00c0: 00 00 00 00 50 41 45 31-31 34 31 30 33 30 30 30 ....PAE114103000
00d0: 38 37 36 20 31 34 30 33-30 35 20 20 00 00 00 c2 876 140305  ....
00e0: 31 31 31 30 34 30 39 30-37 31 20 20 20 20 20 20 1110409071
00f0: 20 31 20 20 20 20 20 20-20 20 00 00 00 00 00 00  1        ......
Look familiar? Our monoprice DAC now has identical EEPROM contents, compared to our actual official brocade DAC.

And as expected, after unplugging and replugging the cable, the switch now ID's it as a DAC, and thinks it's an official brocade part:

Code:
telnet@ICX3#show media | include 1/2/[2-5]
Port 1/2/2:  Type : 40GE-Passive Copper
Port 1/2/3:  Type : 40GE-Passive Copper
Port 1/2/4:  Type : 40GE-Passive Copper
Port 1/2/5:  Type : 40GE-Passive Copper

##even more details - still the monoprice DAC
telnet@ICX3#show media e 1/2/2
Port   1/2/2:Type  : 40GBASE-Passive Copper
Vendor Name: BROCADE          Serial Num: PAE114103000876 Revision: A
I wouldn't think this has any effect on functionality, as I said the links worked perfectly fine beforehand. Re-coding it into a brocade part like this was only possible because the monoprice DAC did not have the EEPROM write protect bit set. This exact same procedure can be used to re-code optics to different vendors to bypass vendor locks, but many non-OEM optics have a bit set that prevents EEPROM writing without special unlock strings

A little more information:

The SFP+ EEPROM data location layout, you can see the identity and vendor etc strings are at the beginning (like our QSFP+ monoprice incorrectly had) - http://files.siemon.com/sis/application-guide/sfp-plus-to-sfp-plus_eeprom_summary.pdf

The QSFP+ EEPROM layout, you can see that the vendor and cable type etc now does not start until around the middle. This is where the ICX was trying to read to figure out the cable type, and of course on the monoprice there was nothing there - http://files.siemon.com/sis/application-guide/qsfpplus_to_qsfpplus_eeprom_summary.pdf

Upon closer inspection, this could have potentially caused it refusing to link up, as it could not even read the ident bit to figure out what kind of cable it was. For QSFP modules, it reads byte number 128 (first byte on line 80 in the dump above). This byte is set to "0d" hex to tell the host it's a QSFP+ device. On the monoprice dac, when it read that byte, it was completely empty. You can see in the brocade dump, that bit is correctly set to 0d.

here's a table of possible values for that byte:

https://i.imgur.com/2Id6xp9.png

You can see that if we set that byte to 03 instead, we could make the switch think it's an SFP module. It would probably not like this, given it's a QSFP+ slot (or on second thought, it would just assume it's an SFP+ optic in an SFP+ to QSFP+ adapter)

And upon even further digging, the EEPROM also contains signal attenuation values for the given cable at both 2.5ghz and 5ghz (bytes 186 and 187 respectively), and the host (the switch) uses these values to set up it's transmitter and receiver EQ appropriately. Without it being able to read these on the stock monoprice DAC, the transmitter and receiver probably defaulted to "fallback" values, which could have also resulted in the not quite working links.

Again, copying over the brocade DAC contents should fix this, as it contains these values (4db and 5db, respectively). I'd imagine the attenuation values there are very close to what the monoprice DAC is supposed to have, given they are both the same length, and both 30AWG twinax. Given how much stuff the monoprices come with wrong from factory, I'm surprised they ever worked, especially with cisco equipment (which is what they're advertised for). Thankfully it's not write-locked, so all this is easily fixed with a few commands
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,727
3,075
113
33
fohdeesha.com
Hi,
Is it possible to made this modification with the ICX6450?.

Thanks.
That modification is for as QSFP breakout cable and the ICX6450 has no QSFP ports, so no :) If you want to rewrite an SFP+ optic, 99% of them have the eeprom locked so you won't be able to write anything. The ICX6450 runs linux so it does not have the same commands, the only I2C write commands I remember are "dm i2c-write" but it only takes one byte at a time so you'll have to run that ~100 times and you'll need to find out the I2C ID of the optic as well. Don't have one powered on at the moment to test
 

am45931472

Member
Feb 26, 2019
87
17
8
Is it possible to manually change the fan speed on the Brocade Ruckus 7150-24p? the ruckus website seems to say that fans can only be automatically controlled. I've tried fan-speed via CLI and it is not recognized as a command. I've swapped out the stock fans for some noctua ones going from an audible but quiet switch to a basically silent switch. Temps are good-ok, between 65-75c, but I think even at full speed these noctua fans would be near silent. 72c seems to be the temp threshold for high speed.
 

microserf

New Member
May 20, 2019
20
13
3
Code:
ICX7150-Boot>nand info

Device 0: nand0, sector size 1024 KiB, Micron NAND 2GiB
  Page size       4096 b
  OOB size         224 b
  Erase size   1048576 b
  subpagesize     4096 b
  options     0x   10200
  bbt options 0x       0
ICX7150-Boot>nand bad

Device 0 bad blocks:
  00a00000
  05a00000
  05b00000
Gah.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,727
3,075
113
33
fohdeesha.com
Code:
ICX7150-Boot>nand info

Device 0: nand0, sector size 1024 KiB, Micron NAND 2GiB
  Page size       4096 b
  OOB size         224 b
  Erase size   1048576 b
  subpagesize     4096 b
  options     0x   10200
  bbt options 0x       0
ICX7150-Boot>nand bad

Device 0 bad blocks:
  00a00000
  05a00000
  05b00000
Gah.

you'll be aight
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,727
3,075
113
33
fohdeesha.com
Is it possible to manually change the fan speed on the Brocade Ruckus 7150-24p? the ruckus website seems to say that fans can only be automatically controlled. I've tried fan-speed via CLI and it is not recognized as a command. I've swapped out the stock fans for some noctua ones going from an audible but quiet switch to a basically silent switch. Temps are good-ok, between 65-75c, but I think even at full speed these noctua fans would be near silent. 72c seems to be the temp threshold for high speed.
nope. if it gets too hot it'll ramp the fans up no need to worry about it
 

levi danzer

New Member
Sep 29, 2019
4
1
3
Just a heads up to any one looking. There is a icx6650-56-e-adv on ebay right now. The seller countered with $300.

Debated on picking it up... but I grabbed an sx6036 for my 40gbe needs...

-Levi
 
  • Like
Reactions: fohdeesha

French Chamallow

New Member
Mar 15, 2019
17
3
3
I move slowly but I learn a lot thanks to my ICX-6450 to you all!
My vlans are mounted and I have my ip ranges in each vlan as desired.
intervlan work in the switch.

I start moving vlan 1 devices to all the vlans I've mounted.

I'm having a problem, I have a raspberry with domoticz on vlan 1 and I have a Xiaomi gateway ( 192.168.60.3 ) to handle zigbee devices.

The xiaomi gateway is connected in wifi on a Nano Hd on the IOT vlan.

From pi I can reach the port of the xiaomi Gateway ansd the port is open.

pi@raspberrypi:~ $ sudo nmap -sU 192.168.60.3 -p 9898

Starting Nmap 7.40 ( Nmap: the Network Mapper - Free Security Scanner ) at 2019-11-12 22:36 CET
Nmap scan report for 192.168.60.3
Host is up (0.0021s latency).
PORT STATE SERVICE
9898/udp open monkeycom


Pi don"t receive update information and I can't control zigbee device

I think I have a problem with multicast but I do not know anything about it.
How do I know or make sure that one vlan's multicast arrives on another vlan? sorry if I express myself badly, but I never played with that!
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
I don't know exactly how the Xiaomi zigbee hub works, but as far as I know, multicast in the "consumer" domotic/IoT devices is used mainly for the discovery of the devices, as those communicate using some sort of mDNS query (Chromecast and Apple AirPlay/Bonjour are some notable example of these applications). If that's the case, you cannot route those requests, as those are sent to the multicast address 224.0.0.251 with a TTL=1; if you need to make those devices discoverable between VLANs you'd have to install an mDNS repeater software (say Avahi for example) on a VM/device with an interface on each network you want to listen from or repeat messages to (obviously you can trunk 'em on a single NIC, and make logical tagged interfaces on it).

Proper multicast routing is a whole different beast, and it involves pretty solid knowledge of IGMP, PIM routers and rendez-vous... knowledge that I don't have, unfourtunately, so on this topic I cannot help you much :(.

Googling a bit I've found this document: http://www.netadmin.us/docs/Multicast.pdf
I'm pretty sure that inside the beefy documentation that @fohdeesha has put togheter there are the exact commands and examples that you might need (yea, I've not read it entirely - yet).
 
  • Like
Reactions: French Chamallow

lec668

New Member
Dec 5, 2018
3
4
3
Hello from a new frenchy 7250/48 owner.
160 bucks for a 2 years old swith was (almost) indecent.

@fohdeesha : many thanks for sharing such a massive amout of info with us ! Rare to read such a complete HowTo guide.
Champagne will be offered if you ever come to Paris.

Already thinking about a 40g upgrade !
 
Last edited:
  • Like
Reactions: maes and fohdeesha