Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[infoMatt]

I think this might be the right thread to post this question in,if not please excuse and inform where a more appropriate place would be.

I read in admin user guide for ICX 6610 that one of the limitations of the NTP client and server cannot communicate using hostnames.
I can confirm it will not resolve 0.north-american.pool.ntp.org. If i nslookup the pool domain name i can use one of the IP address it returns just fine.

My question is Is there away around this so i can use the benefits a "pool" has to offer; eg round robin to many servers changing hourly ?

Thanks

This is a limitation that I've encountered in many switches, as they tend to not implement a DNS client, or using it sparingly...
Basicly, you can insert many servers, or if you have a raspberry pi or a virtualization server, install a NTP server on these and use it for your network devices.
But as far as I know, the clock is used "only" for the timestamp of the messages, if the clock is wrong nothing serious happens... it would continue to switch and route packets just fine.

 

[infoMatt]

Hello all,
[...]
Now here's the issue - No matter what I do - I just can't seem to get the boot process to drop into the boot monitor prompt. I have tried continuously hitting the 'b' key before and through powering up the device, hitting it just after the $ prompt but before the ICX Boot .... displays, right after the Enter 'a' to stop at memory test etc. But somehow I just don't seem to be hitting it at the right time. I've tired this at least 2 dozen times over the past couple of days, but it just does not seem to work.

When the switch boots, does it recognise your inputs? ie., can you execute commands at management prompt?

If it doesn't work also with the switch OS loaded, it might be a faulty cable or a hardware flow control limitation... it's strange, no doubt about it...

If the test succeeds, issue a reload and keep 'B' down

EDIT: try also running minicom as superuser, in case it's a strange issue with user permissions on the serial device.

 

[Wolfstar]

Found it: https://forums.servethehome.com/ind...be-40gbe-switching.21107/page-117#post-240889

That's for stacking two 6610s using the breakout ports; I'm trying (hoping? I guess?) to stack an ICX6450 with an ICX6610 in a mixed stack using the breakout cables. Looks like it's not possible; as I said the only post I found was Fohdeesha saying you have to use the front 10G ports.

 

[Wolfstar]

Hello all,

<Snip!>

I'm sure I'm missing something very simple. But I just can't seem to get the timing right - or am I missing something else. Trust me, I'm really embarrassed to be asking for help on something that should be so simple. But at this time I am out of options here and am on the verge to returning the unit as defective because no matter what I do - I can't seem to get to the monitor prompt. I keeps wanting to boot all the way through no matter when I hit the 'b' key. The Boot Code version is 10.1.00 (grz10100).

Please heeellllpppp - getting really desperate.

Try using screen instead of minicom in case it's something with minicom. From command prompt:

sudo screen /dev/ttyS0

You'll need to hit Ctrl+A, then K to close ("kill") the Screen session. (You can let go of Ctrl+A before the K.)

See if that lets you hit the 'b' to interrupt the boot process.

 

[infoMatt]

That's for stacking two 6610s using the breakout ports; I'm trying (hoping? I guess?) to stack an ICX6450 with an ICX6610 in a mixed stack using the breakout cables. Looks like it's not possible; as I said the only post I found was Fohdeesha saying you have to use the front 10G ports.

That shoud work the same... use the 4 breakout for the 6610 and the 4 10G for the 6450... am I wrong?

 

[Wolfstar]

That shoud work the same... use the 4 breakout for the 6610 and the 4 10G for the 6450... am I wrong?

I'm trying to stack one 6610 with one 6450. Apparently, you can't use the QSFP breakout to do this, even though I want to stack using 4 ports. So, I'd have to run from front ports, which ends up messy when I'm physically separating them.

It is what it is, but it'd be nice to avoid the issue. My cable management is bad enough as it is.

 

[groove]

Try using screen instead of minicom in case it's something with minicom. From command prompt:

sudo screen /dev/ttyS0

You'll need to hit Ctrl+A, then K to close ("kill") the Screen session. (You can let go of Ctrl+A before the K.)

See if that lets you hit the 'b' to interrupt the boot process.

Thanks for the tip.

Unfortunately, it behaves the same through screen as well. Even tried picocom. The really frustrating thing is that the same setup works just fine with an SX6012.

 

[fohdeesha]

Yep, @fohdeesha answered a very similar question a couple of days ago, with the correct procedure and commands to use the 40Gbreakout as the stack backbone... try look at the last few pages of this thread

well, he's asking about using the breakout ports for a MIXED stack (eg to a ICX6450) and I'm about 99% sure it won't let you do that. There's some mixed stacking port commands that I'm pretty sure won't even let you use the breakout port numbers as an argument

keep in mind mixed stacking with an ICX6450 and a 6610, the 6450 basically just acts as a port extender for a 6610. Everything works great and you get the unified CLI, but the 6450 offloads all the l3 work to the 6610, and if you lose the 6610, the 6450 will no longer do much if anything. It's not an HA thing like two 6610s

 

[fohdeesha]

Hello all,

Really need some help here - Just got delivery of a preowned ICX 6610-48P-E from ebay. Thought I'd use the weekend to update the firmware and configure it for network - noobie here using a L3 capable switch - so thought I'd use this switch to accomplish 3 goals at once - upgrade to 10GBE, power up a few POE devices and also learn more about L3 level routing and protocols in general.

Unfortunately, have hit into a snag right on the get-go just upgrading the firmware - at my wits end trying to figure out how to get to the monitor prompt to do a factory reset of the switch - So here's my setup - I have a serial cable connected to the serial io RJ45 port on the switch (IOIOI) to the serial port (DB9) of an X9drd-7ln4f motherboard running a vanilla install of ubuntu 18.04 - server. I am using minicom to communicate with the switch over the serial port. I've configured minicom as follows :

Serial Device : /dev/ttyS0
Lockfile Location : /dev/lock
Bps/Par/Bit : 9600 8N1
Hardware Flow Control : No
Software Flow Control : No

Now here's the issue - No matter what I do - I just can't seem to get the boot process to drop into the boot monitor prompt. I have tried continuously hitting the 'b' key before and through powering up the device, hitting it just after the $ prompt but before the ICX Boot .... displays, right after the Enter 'a' to stop at memory test etc. But somehow I just don't seem to be hitting it at the right time. I've tired this at least 2 dozen times over the past couple of days, but it just does not seem to work.

I'm sure I'm missing something very simple. But I just can't seem to get the timing right - or am I missing something else. Trust me, I'm really embarrassed to be asking for help on something that should be so simple. But at this time I am out of options here and am on the verge to returning the unit as defective because no matter what I do - I can't seem to get to the monitor prompt. I keeps wanting to boot all the way through no matter when I hit the 'b' key. The Boot Code version is 10.1.00 (grz10100).

Please heeellllpppp - getting really desperate.

the most important question here: once it fully boots and drops down to the switch prompt, can you type stuff and see it show up? eg can you run "show version"? If not (and I suspect not), the switch is defective with broken/burnt out serial input, and I would return it. I would say it could potentially be the cable but you state it works fine elsewhere so it's almost certainly a bad serial input on the switch

 

[fohdeesha]

little easter egg I found while going through some bootloader source, an ICX7230 product that was never released:

+#define CONFIG_SYS_PROMPT     (is_platform_sica_24L() ? "ICX7230-Boot> " : (is_platform_sica() ? "ICX7250-Boot> " : "ICX7450-Boot> "))

 

[fohdeesha]

I have a question about what looks like read errors on the Flash.

Screenie below is from an overnight Minicom session to a 6430-24 (as you might be able to tell from the prompt!).

Should I be worried ? Should I run an fsck at the boot prompt? Pray to @fohdeesha?
View attachment 12195

sounds like the flash might be failing. To be sure I would follow the update guide again to completely lay down a fresh bootloader and OS image again and make sure you're running the latest (in the ICX64xx series both the bootloader and OS live in the same SPI flash). I would do the factory-default command again too to remove any config files and logs. If you still get read errors, it's probably the flash

it could just be the physical flash is fine but something got corrupted during a power pull or something, the sdb instead of sda makes me think the icx6430 splits it up into two flash devices, one for the bootloader and OS image and another for configuration files and logs, the ICX6450 does this (covered in the architecture brief pdf). If you can follow the accessing linux guide and try some fsck commands go for it. If you do, lemme know what you find with sda vs sdb (I'm curious why there's more than one when it only has one physical flash)

 

[fohdeesha]

after poking through linux on my 6450 (which runs the same bootloader and OS image as your 6430) I don't see references to any sd* devices at all, just the jffs2 filesystem and the permanent block mount for config files:

/ # df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mtdblock3           32768       956     31812   3% /fast_iron

/dev # ls
console    mtd3       mtdblock1  null       ptyp4      tty        ttyp5
mtd0       mtd3ro     mtdblock2  ptmx       ptyp5      ttyS0      ttyp6
mtd0ro     mtd4       mtdblock3  pts        ptyp6      ttyp0      ttyp7
mtd1       mtd4ro     mtdblock4  ptyp0      ptyp7      ttyp1      ttyp8
mtd1ro     mtd5       mtdblock5  ptyp1      ptyp8      ttyp2      ttyp9
mtd2       mtd5ro     mvPP       ptyp2      ptyp9      ttyp3      zero
mtd2ro     mtdblock0  net        ptyp3      random     ttyp4

so I'm either blanking on something obvious, or it's very odd you're getting FAT filesystem errors on an sdb device I didn't know existed. Did you follow the update guide? you're sure you're not running some weird dev firmware from ebay?

 

[sean]

sounds like the flash might be failing. To be sure I would follow the update guide again to completely lay down a fresh bootloader and OS image again and make sure you're running the latest (in the ICX64xx series both the bootloader and OS live in the same SPI flash). I would do the factory-default command again too to remove any config files and logs. If you still get read errors, it's probably the flash

it could just be the physical flash is fine but something got corrupted during a power pull or something, the sdb instead of sda makes me think the icx6430 splits it up into two flash devices, one for the bootloader and OS image and another for configuration files and logs, the ICX6450 does this (covered in the architecture brief pdf). If you can follow the accessing linux guide and try some fsck commands go for it. If you do, lemme know what you find with sda vs sdb (I'm curious why there's more than one when it only has one physical flash)

My read was that it's the computer running minicom. A sdX device would be from SATA while the 6450, and I assume 6430, boot off the initramfs and mount /dev/mtdblock3 to /fast_iron.

 

[fohdeesha]

OK, maybe a much simpler explanation: those FAT-fs sdb1 read errors are from the actual linux box you're running minicom on, not the switch itself Does that linux box have an sdb1 device with a FAT file system?

 

[fohdeesha]

My read was that it's the computer running minicom. A sdX device would be from SATA while the 6450, and I assume 6430, boot off the initramfs and mount /dev/mtdblock3 to /fast_iron.

nice timing, just came to that obvious conclusion finally as well after my chili coma wore off

 

[tojoski]

Does anyone have a 10gig Ethernet transceiver that they recommend for the 6450?

I would go with just dac cables but my board already has dual 10g Ethenet.

 

[Wolfstar]

Does anyone have a 10gig Ethernet transceiver that they recommend for the 6450?

I would go with just dac cables but my board already has dual 10g Ethenet.

Fiberstore has some that can be coded any which way and are relatively cheap. Just remember that you're talking a much shorter run than normal for Cat6/6a with those.

 

[fohdeesha]

either of these work fine

https://www.amazon.com/Mikrotik-S-RJ10-MikroTik/dp/B078SNK1MY

Brocade 10G-SFPP-T Compatible 10GBASE-T SFP+ Copper RJ-45 30m Transceiver Module

 

[infoMatt]

well, he's asking about using the breakout ports for a MIXED stack (eg to a ICX6450) and I'm about 99% sure it won't let you do that. There's some mixed stacking port commands that I'm pretty sure won't even let you use the breakout port numbers as an argument

My bad, sorry... I was thinking that the command used to bring up the stack would have been the same...

 

[fohdeesha]

My bad, sorry... I was thinking that the command used to bring up the stack would have been the same...

I would absolutely kill for the full sourcecode to the fastiron binary, because I know changes like that would be one or two lines of changes max. Not to mention the fan control logic and everything else requested in this thread

If there's any ruckus employees reading this: I will pay, REALLY well