Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

Alright, I have no idea whether I am doing something wrong, or that all of my RS232 hardware is faulty, however I am unable to get any console output with the CH340G USB to RS232 adapter and an RS232 to RJ-45 adapter, even with the Cisco router (the CH340G USB to RJ45 cable is working fine with the same Cisco router, however I am getting no output from the Brocade switch with it).

Add to that the fact that I have never been able to get serial over RS232 working with any hardware (I have successfully used serial over USB directly to contacts on boards, and now to RJ-45), and I have absolutely no idea anymore.

you're not plugging the rj45 serial cable into the fcx are you? the fcx has no rj45 serial ports, the rj45 port on the back is an ethernet port

 

[Just2Bad]

Hello one quick question on th eicx6610 and the setup I want to do. , if I have 2x icx6610 and I have 4 proxmox hosts that I want to connect at 1x40gbe.

is it possible to stack the 2 switch via the 2 ports that can be use as break out but at full 1x40gbe in stkaing mode and use non breakout port to connect to the proxmox hosts ?

I want full 1x 40gbe to the host I want to avoid bond on the proxmox hosts cause you never get the 40gbe form a single thread connection



or would I end up with 4x10 + 4x10 for the staking link on the right ? can those be 1x40 + 1x40 in staking mode if there's one ?

 

[fohdeesha]

Hello one quick question on th eicx6610 and the setup I want to do. , if I have 2x icx6610 and I have 4 proxmox hosts that I want to connect at 1x40gbe.

is it possible to stack the 2 switch via the 2 ports that can be use as break out but at full 1x40gbe in stkaing mode and use non breakout port to connect to the proxmox hosts ?

I want full 1x 40gbe to the host I want to avoid bond on the proxmox hosts cause you never get the 40gbe form a single thread connection

View attachment 28153

or would I end up with 4x10 + 4x10 for the staking link on the right ? can those be 1x40 + 1x40 in staking mode if there's one ?

as the guide said, those two ports are fixed 4x10gbe, even in stacking mode

 

[Just2Bad]

as the guide said, those two ports are fixed 4x10gbe, even in stacking mode

thanks I wanted to be sure and have a confirmation form someone else before doing the move and buy a 2nd one, I'll will find another solution

 

[NablaSquaredG]

is it possible to stack the 2 switch via the 2 ports that can be use as break out but at full 1x40gbe in stkaing mode and use non breakout port to connect to the proxmox hosts ?

If you find out how, let me know. I've tried it multiple times, but never got it to work. I doubt it is possible.

You can either stack using the 40G non-breakout ports, or the 40G non-breakout + 40G breakout together, but not only with the 40G breakout ports... Or maybe I'm stupid?

 

[itronin]

You can either stack using the 40G non-breakout ports, or the 40G non-breakout + 40G breakout together, but not only with the 40G breakout ports... Or maybe I'm stupid?

I have only tried a two switch setup and I have only been successful with the first two scenarios. I may also be stupid though - WK?

 

[Just2Bad]

we all stupid lolll, but thanks for feedback now I'm sure it's not possible
....

 

[NablaSquaredG]

I still hope that @fohdeesha will do some magic and come up with a guide that allows the third scenario (stacking only with the 40G breakout ports)

 

[Just2Bad]

I still hope that @fohdeesha will do some magic and come up with a guide that allows the third scenario (stacking only with the 40G breakout ports)

would have been nice for sure, now I have to find another switch with more 40gbe ports to stake with my 6610

 

[DerTom]

Hi!

During configuration of my new ICX6610 I do get the following error:

ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
.$
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
BOOT INFO: load monitor from boot flash, cksum = 71f1
BOOT INFO: verify flash files.........
error in reading i2c device 1
eeprom_read: reading i2c device 1 failed
error in writing i2c device 1, error code: 5
eeprom_write: writing i2c device 1 failed
error in reading i2c device 1
ONBOARD_EEPROM read failure 1.
Backplane MAC address is not configured.
Use default 0004.80a0.4000
BOOT INFO: load image from primary copy...

error in reading i2c device 1, error code 7
eeprom_read: reading i2c device 1 failed
error in writing i2c device 1, error code: 5
eeprom_write: writing i2c device 1 failed
platform type = 12
PCIE-1 LTSSM status: 22
PCIE Switch status: 0
..............................
Firmware integrity checksum passed
.......
Starting Main Task .------------------------------------------------------------------
M:9 L:0 - chow_led_ch_mux_enable, error in seting up mux
------------------------------------------------------------------
M:9 L:0 - chow_led_ch_mux_enable, error in seting up mux
..CPSS DxCh Version: cpss3.4p1 release
------------------------------------------------------------------
M:9 L:0 - PS EEPROM I2C Open: i2c cmd failed, error 7, ps 1
------------------------------------------------------------------
M:9 L:0 - PS EEPROM Offset Read: eeprom mux open error, ps 2
Fatal Error: No FANTRAY fan is present or fan has failed
System is shutting down!!!
Rebooting(0)...

The device is on a bootloop...

The led of the power supply is on but the fan is still 'spinning'!? As there is the message 'No FANTRAY fan present' I assume there is something wrong with the fan... Is this correct?

And most of all which way do I get the device to boot properly?

Kind regards,

tom

 

[frogtech]

Checking in after a while, has anyone made a final determination on the best fans for icx7250 24 port poe? I've tried a couple diff but they're still the loudest item in my rack. The hard part is keeping the switch below 60c still. I think the best solution someone posted was using a heat pipe and some extra heatsink attached.

 

[MannyLNJ]

I recently acquired a FCX648S-HPOE and need to know what is the correct part to get to add a QSFP+ Port so I can add 2.5GB Ethernet ports? I already purchased a XFP module which I believe is the wrong item

 

[DerTom]

Hi!

During configuration of my new ICX6610 I do get the following error:



The device is on a bootloop...

The led of the power supply is on but the fan is still 'spinning'!? As there is the message 'No FANTRAY fan present' I assume there is something wrong with the fan... Is this correct?

And most of all which way do I get the device to boot properly?

Kind regards,

tom

I do have replaced the power supply but there is still this error and the device on a bootloop... Mr. Google doesn't seem to know how to get rid of this.

If you have any hints, let me know, please.

 

[msg7086]

Just bought a 6450-48p on ebay with a $100 offer. They seem to have more than 100 units available. I'm not linking to it to reduce STH effect but it should be very easy to find. In case anyone is looking to buy one.

 

[oxoyoz]

Hi I got an ICX6610-48 recently and want to know:

1. Is it beneficial to upgrade its DDR2 memory from 512M to 1G/2G stick? These memory sell for 5 bucks on Ebay including shipping. But I am not sure it’s better used for a beer instead.

2. How much idle power can it reduce if I remove its 40gbps QSFP module on its back? I don’t have a use case for it yet. Granted I would lose the MGMT ports too but after initial setup I can SSH on the front ports to access it. Trying to save on utility bills a bit.

Thanks.

 

[NablaSquaredG]

These memory sell for 5 bucks on Ebay including shipping.

Are you sure? iirc the DIMMs are ECC SODIMMs and they're very hard to come by (I contemplated a memory upgrade for my SX6036)
NVM, according to @fohdeesha they're RDIMMs, and some pages ago he posted a link to this offer Centon Lot of 2 RD703G01 2GB PC2-5300 Reg ECC CL5 DDR2 667Mhz 1 Rank SODIMM 6-2 | eBay

2. How much idle power can it reduce if I remove its 40gbps QSFP module on its back? I don’t have a use case for it yet. Granted I would lose the MGMT ports too but after initial setup I can SSH on the front ports to access it. Trying to save on utility bills a bit.

Just a wild guess, but probably not much (if any power saving at all). Maybe it would make more sense to get an ICX6450-48 and sell the ICX6610?

 

[juju]

I'd like to start hosting a website from my homelab. I am planning to put all publicly available websites into a separate vlan, then block it off using acl rules. Is this the right approach ? I'd like help with the acl rules . What should it be? I am also looking at using tailscale but want to get the vlan segmented off first. I have an icx7250.

Thanks

 

[msg7086]

I'd like to start hosting a website from my homelab. I am planning to put all publicly available websites into a separate vlan, then block it off using acl rules. Is this the right approach ? I'd like help with the acl rules . What should it be? I am also looking at using tailscale but want to get the vlan segmented off first. I have an icx7250.

Thanks

I wouldn't do this as this sounds a bit overkill, but you definitely can. I would rather put all websites (assuming VMs) behind a front end gateway, and do ACL/proxy on the gateway.

 

[juju]

I wouldn't do this as this sounds a bit overkill, but you definitely can. I would rather put all websites (assuming VMs) behind a front end gateway, and do ACL/proxy on the gateway.

Can you elaborate a bit? What do you mean by a front end gateway? Traefik? HAProxy? Jump server? It seem rather cleaner to just put them all in their own vlan and block them off from the rest of the home network, no? Happy to learn the merits of what you are proposing.

 

[msg7086]

HAProxy or something like that (I use nginx). Do you announce static IP range through your ISP? Because if you only have 1 IP and serve multiple websites I imagine you'd use some kind of front end proxy to route traffic?