Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[hmw]

Upgraded both the ICX-6610-24PE and ICX-6610-48PE to 08.0.30u. Now waiting to grab some licenses from Da Man.

The switches came with two PSU Rev B for one switch and a Rev B and Rev C for the other switch. Was worried about getting Rev A, even the noise on the Rev B after they spin down - is quite loud

I get the following message after boot on one of the switches (tried swapping the PSUs and fan trays around - but only tried with one PSU at a time):

Power supply 1  detected in standby mode.

Power supply 2  detected.

Power supply 2  detected.
Power supply 2  is up.
PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Info: Adding new 55V capacity of 748000 mW, total capacity is 748000, total free capacity is 748000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
lost arbitration, dev=50
i2c controller reset, dev=50
lost arbitration, dev=50
i2c controller reset, dev=50
lost arbitration, dev=50
i2c controller reset, dev=50
lost arbitration, dev=50
i2c controller reset, dev=50
lost arbitration, dev=50
i2c controller reset, dev=50
M:9 L:0 - PS EEPROM Offset Read: i2c cmd failed, i2c addr 0x50 error 4, ps 1
------------------------------------------------------------------
lost arbitration, dev=50
i2c controller reset, dev=50
PoE Info: PoE module 1 of Unit 1 initialization is done.

Is the "i2c cmd failed message" normal?

 

[infoMatt]

Is the "i2c cmd failed message" normal?

I'm not sure but I'd say no, that's not the expected behaviour. It seems it struggles to read the PoE module EEPROM; can you enable power and confirm it is powering up some PD?

 

[nthu9280]

Next step - it's finally time to pull the trigger on a rack cabinet...!

Below are some tips when you are in the market for a rack in US.
- if you live near a large metro area, setup alerts on craigslist. Full height 42u are more common than 24u and shorter.
- Also make sure it is full depth (1070mm) or close to it. (Speaking from experience )
- Square holes.

 

[Rand__]

- Also make sure it is full depth (1070mm) or close to it. (Speaking from experience )

Totally off topic, but recently got a full depth rack and actually couldn't fit any of my SM Rails in it
I have short and regular ones, but none have the required length

 

[ljvb]

Totally off topic, but recently got a full depth rack and actually couldn't fit any of my SM Rails in it
I have short and regular ones, but none have the required length

I have that problem with the rack I am trying to give away.. fits my SM crap (before I got rid of the SM stuff, but all the DL 360 and 380s are too long.

 

[deified]

Anyone have any "best practices" for the Brocade ICX6610 48p?
Here is a little about my setup;
Windows 10 machine with a MCX354A-FCBT --> Cisco QSFP-40G-SR-BD Compatible [fs.com brand] --> 15m om5 LC to LC --> Cisco QSFP-40G-SR-BD Compatible --> ICX6610 port 1/2/1.
Server 2019 with a MCX354A-FCBT --> Mellanox MC2207130-002 56gb/s DAC --> ICX6610 port 1/2/6
I get about 30-35 Gbps running 3 iperf3 tests at once.
I wasnt sure about enabling all ports poe, does this increase overall power consumption?
thanks for any helpful tips or tricks

 

[Schut]

I'm all set up with a 48 port POE ICX-6610, and I'm extremely pleased for the $150 shipped I paid to Ebay. I've thanked Fohdeesha privately, but I'll do so again here publicly. Many, many thanks.

Here's my question:

I have maybe 10 POE phones, 7 POE cameras (some external with IR), and 3 Unify POE wifi APs. Since this is my first layer 3 switch, I have yet to get VLANs working for all these devices, although I intend to, and when I do I expect I will segregate each class of port into its own bank on the switch.

In the meantime, is there any reason NOT to enable POE on all ports? Is this harder on the power section of the switch, even if I don't have all the ports loaded with POE devices? I notice the default from Brocade is to disable them, so I wonder if that is suggestive of anything.

 

[klui]

@hmw
Honestly the dented screw looks fine. Couldn't pliers be used to bend it back? Just put a piece of cloth between to keep it from getting scratched. Download the offical release notes to see closed defects and known issues. My 6610-48P spews out the same I2C errors when only one PSU is powered up and both are plugged in--the error would refer to the PSU that's not powered up. Power up both PSUs and they should go away--if not you have a problem.

@deified
The best practices depend on what you want your network to do. @fohdeesha gave you a good start with the guide for login credentials. Other things to think about would be enabling spanning tree, and DHCP snooping. If you want to go hardcore, consider port security, DoS protection, MAC authentication, TACACS, RADIUS, and disable all ports unless they're used.

@Schut
My recommendation is turn off PoE unless you know the port is connected to a device that uses it. I've found some devices can fool PoE negotiation on the 6610, but that was on old firmware. This practice falls in line with "only turn on services if you use them and disable by default." Power wise it shouldn't make any difference.

 

[hmw]

@hmw
Download the offical release notes to see closed defects and known issues.

Enhancements in FastIron 08.0.30u
FastIron 08.0.30u release contains defect fixes. There are no enhancements in this release.

Yeah downloaded them and that’s all that the official release notes say

updated anyways

 

[klui]

Yeah downloaded them and that’s all that the official release notes say updated anyways

Look again. I made the same mistake in the past. For a lot of enterprise release notes enhancements, fixes, and known issues are in separate sections. That entry you quoted is on page 6 (of 226). Scroll up 2 pages before that and you will see "Closed defects with code changes in Release 08.0.30u" on page 29 and "Known Issues in Release 08.0.30u" on page 34.

 

[hmw]

Look again. I made the same mistake in the past. For a lot of enterprise release notes enhancements, fixes, and known issues are in separate sections. That entry you quoted is on page 6 (of 226). Scroll up 2 pages before that and you will see "Closed defects with code changes in Release 08.0.30u" on page 29 and "Known Issues in Release 08.0.30u" on page 34.

Awesome - thanks, looked again and found the known issues and defects closed !

 

[Reece5646]

Good day all, new here, so my apologies if I’m posting in the wrong location ...

Question, after following these instructions, all went well, however the switch is now in router mode, and I can’t figure out for the life of me how to get it back to switch mode

Any help would be great !

 

[infoMatt]

I have that problem with the rack I am trying to give away.. fits my SM crap (before I got rid of the SM stuff, but all the DL 360 and 380s are too long.

Most of the times you can adjust the posts, moving them forward or backward untill you find a spot where all the rails align and can snap into the mounting holes. It's fine if the tail of the server is behind the rear post, if you need it.

Good day all, new here, so my apologies if I’m posting in the wrong location ...

Question, after following these instructions, all went well, however the switch is now in router mode, and I can’t figure out for the life of me how to get it back to switch mode

Any help would be great !

What do you mean? If you don't define any VE on a VLAN it will just act as a "pure" L2 device.

 

[Spearfoot]

Good day all, new here, so my apologies if I’m posting in the wrong location ...

Question, after following these instructions, all went well, however the switch is now in router mode, and I can’t figure out for the life of me how to get it back to switch mode

Any help would be great !

The CLI wizards here will probably scoff at my simple approach of using the Web administrative interface for doing this. But I'm pretty sure you can fix this by loading the Web GUI and insuring 'L2 Switching' is enabled on the General System Configuration page, and 'Route Only' is disabled on its 'Advanced Settings' page.

 

[Reece5646]

The CLI wizards here will probably scoff at my simple approach of using the Web administrative interface for doing this. But I'm pretty sure you can fix this by loading the Web GUI and insuring 'L2 Switching' is enabled on the General System Configuration page, and 'Route Only' is disabled on its 'Advanced Settings' page.View attachment 14544
View attachment 14545

THANK YOU !
That’s exactly what I was looking for and that was right in front of my face this entire time this is why I should not be doing this stuff at 4 o’clock in the morning lol

thank you a million !

I will definitely be testing this in a few hours when I get some motivation LOL

 

[Reece5646]

The CLI wizards here will probably scoff at my simple approach of using the Web administrative interface for doing this. But I'm pretty sure you can fix this by loading the Web GUI and insuring 'L2 Switching' is enabled on the General System Configuration page, and 'Route Only' is disabled on its 'Advanced Settings' page.View attachment 14544
View attachment 14545

 

[Reece5646]

Unfortunately my settings were the same as yours dammit lol

what I am seeing is
When connecting through putty

ICX6610-24 Router#

I use to see

ICX6610-24 Switch#

unfortunately what you suggested was already done :-(

I know there’s two different modes for the switch after upgrading the firmware I now have it in router mode, that I’m looking to change back to switch mode and I can’t find documentation anywhere. unless of course I’m understanding something incorrectly.

The reason why I want to put it back to switch mode is under router mode it doesn’t function the way I need it.
until I did the firmware update it was working perfectly so I guess in fact I most likely should’ve done the firmware update.

for example:

can’t set IP address / Gateway etc,
all I get is invalid input, with almost every command I put in, so the inputs must be different under router mode, I’m assuming.

 

[hmw]

I know there’s two different modes for the switch after upgrading the firmware I now have it in router mode, that I’m looking to change back to switch mode and I can’t find documentation anywhere. unless of course I’m understanding something incorrectly.

Hmm - if you download the firmware from Brocade, there's - for example - the FCXS08030u.bin and the FCXR08030u.bin firmwares in 'Images'. The 'S' firmware is the switch firmware that is smaller and drops ALL L3 code. You can switch back and forth between these two images. Is this what you're referring to?

 

[klui]

ICX6610-24 Router#

I use to see

ICX6610-24 Switch#
.
.
.
can’t set IP address / Gateway etc, all I get is invalid input, with almost every command I put in, so the inputs must be different under router mode, I’m assuming.

Router code is more flexible and I would prefer to use that. In your old case there was a global IP address assigned for the switch's management switch through what I presume is ip address ....

What you could do on router code is

interface management 1​

ip address <IP> <mask>​

The above command will set your out-of-band management interface to what you want. You can also have IP addresses associated with your VLANs and you have access to those inband.

 

[Reece5646]

Hmm - if you download the firmware from Brocade, there's - for example - the FCXS08030u.bin and the FCXR08030u.bin firmwares in 'Images'. The 'S' firmware is the switch firmware that is smaller and drops ALL L3 code. You can switch back and forth between these two images. Is this what you're referring to?