Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[jei]

Yes been through the documentation, and does not explicitly say recommended. And in case it is recommended over 09 I would like to know why.
Anyways I'll try my luck on 09, have room for some trial, error and tinkering.

It is recommended by Ruckus. 09 is not considered stable.

Ruckus ICX software currently has two recommended release types; Stability and Technology.

Stability Release: This is for customers where stability is of utmost importance. This release may not contain every feature available for your product.

Technology Release: This is for customers looking to utilize the maximum feature set available for your product.

We recommend most customers utilize the Stability Release if it contains all needed features for your network. A Technology Release is recommended if your network requires newer features not available in the Stability Release.

 

[Sealside]

I do like the webgui in 09. Even though you configure and use the cli, it is nice to have a webui to get an overview.



Another question is, my firmware update failed for the boot

SH@beef1>show version
  Copyright (c) Ruckus Networks, Inc. All rights reserved.
    UNIT 1: compiled on Jan 12 2024 at 09:59:12 labeled as SPR09010j
      (33554432 bytes) from Primary SPR09010j.bin (UFI)
        SW: Version 09.0.10jT213
      Compressed Primary Boot Code size = 786944, Version:10.1.26T215 (spz10126)
       Compiled on Tue Nov 29 15:43:15 2022

spz10126 is probably not the same what's included in the SPR09010-release. I fail when I try to update it using manifest, any ideas how I can update it?

Best Regards, S

 

[Sealside]

Trying to get the 7250 to pass the WAF, I had to do something with the fans.
I went with the same approach as https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-331607

3xMechatronic MR4020C12B1-RSR and 1x M5210E12C-RSR
I reused extension cables from my previous Noctua fans to make connectors for the fans.
I used solder seal wire connectors to attach the connectors with a heat gun.

The noise level is more than my Juniper Ex3300 (Which has 3xNoctuas), but it is ok and a lot better than Stock fans.
Noctua fan are delivered with noise adapters, probably could use those on stock, but I have not tested.

 

[Vesalius]

Another question is, my firmware update failed for the boot

SH@beef1>show version
  Copyright (c) Ruckus Networks, Inc. All rights reserved.
    UNIT 1: compiled on Jan 12 2024 at 09:59:12 labeled as SPR09010j
      (33554432 bytes) from Primary SPR09010j.bin (UFI)
        SW: Version 09.0.10jT213
      Compressed Primary Boot Code size = 786944, Version:10.1.26T215 (spz10126)
       Compiled on Tue Nov 29 15:43:15 2022

spz10126 is probably not the same what's included in the SPR09010-release. I fail when I try to update it using manifest, any ideas how I can update it?

Best Regards, S

I have a similar Boot Code version on an icx-7150-zp using SPR09010j.bin (UFI), Version:10.1.26T225 (mnz10126), curious why you think this incorrect? BTW Ruckus released a minor update RUCKUS ICX FastIron 09.0.10j_cd1 (GA) Software Release (.zip) you can try.

 

[Jason Antes]

For those wondering about 2.5Gb connectivity, I recently bought 2 FS.com Intel coded 10GbaseT transceivers to use with my 6610. 1 works and the other does not (due to the branding encoding being botched). At any rate, the one that does work connects in as 10Gb and allows my desktop to connect at 2.5Gb. SFP-10G-T. I've been doing large amounts of file transfers without issues (consistent copies across network to my NAS server to the tune of 8-10 hours straight) and it seems to be stable.

 

[abster]

I was able to get both servers connected by assigning fixed IP's to both. Once I was able to get the link going on Unraid, I added to network connect and it's working fine at 10GB speeds. In Promox, assigning a fixed IP allowed the link to work but I am not getting DNS name resolution.

I'll look in DAC cables instead of the SFP transceivers.

Thanks!

Given the issues I'd suggest reducing the complexity of your testing and going component by component.

from your post it sounds like you tested 1gbe into your server nics and used some of the 1/1 ports and that worked but I'd go ahead and make sure 10gb is working.

test the servers 10gbe back to back which will require assigning a different subnet than you have on your LAN and static IP'ing the servers. If you don't want to mess with your unraid and prox configurations you should be able to boot a live ubuntu or something like that for the testing.

Use the same network cable that you were running between the NIC 10gb-t interface and the copper SFP+ to run back to back. ping test and if you can move some data back and forth.

Not sure which DELL broadcom those are but I have had challenges in the past with some of them if they are set up for virtual interfaces inside the NIC. fixing that required going into the iDRAC then into the card to nuke the configuration. quick google makes me think those are NOT the virtual NIC broadcom's...

I'm not an expert with prox but if the 10gb interface came up and was configured for VM's then I would not expect the prox host to necessarily be trying to pull DHCP.

On Unraid, should just be a "tick box" to configure DHCP on that interface.

I'm going to make an observation. running a pair of 10gb-t SFP+ adapters is going to make that SFP cage area VERY VERY WARM if not VERY HOT. If you have the funds and the distance works - really suggest DAC and changing out your NIC's. If you have the funds but distance is an issue then change your NICs and 10Gb MM optics and fiber patch.

 

[kpfleming]

I was able to get both servers connected by assigning fixed IP's to both. Once I was able to get the link going on Unraid, I added to network connect and it's working fine at 10GB speeds. In Promox, assigning a fixed IP allowed the link to work but I am not getting DNS name resolution.

I'll look in DAC cables instead of the SFP transceivers.

Thanks!

If you have connectivity using statically assigned addresses then the transceivers/cables are not the problem. It sounds like those machines are on a VLAN which does not have access to (or does not include) your DHCP/DNS/etc. server.

 

[jei]

Anyone know where I can get rack ears? Not sure if they all use the same but specifically for a ICX7250 in case they're different.

Been looking for ICX7000-RMK too for a while. Mostly it would cost the same to buy a cheaper model of 7250/7450 with ears than just ICX7000-RMK set.

 

[Beaflag VonRathburg]

EDIT: That was an adventure. Received two switches from a site update. The 6450-48p updated without an issue and works fantastically. The 7150-48 on the other hand had all kinds of odd issues. The update kept appearing to work, but it would still boot to the old fimware, reload command would error stating a copy was still in progress, 1 gig ports would auto negotiate to 100m, and more. Eventually most of it was resolved by pulling the power on the switch, running a manifest update to the latest 09.0.10j firmware, and rebooting a few times. Testing afterwards all smooth sailing.

 

[TonyArrr]

Been looking for ICX7000-RMK too for a while. Mostly it would cost the same to buy a cheaper model of 7250/7450 with ears than just ICX7000-RMK set.

Sounds like thar be profit in buying a switch, moving the ears to one’s existing switch, then on selling the new switch…

Wonder what it would take to get stock from Commscope directly…

 

[sbsth]

I changed the default vlan to something else so I could use VLAN 1 (I have Unifi APs). It partially works (devices get an IP via DHCP) but from that VLAN I cannot reach the internet or other VLANs. Did I miss something I needed to clear in configuring VLAN 1 to make it routable?

 

[kpfleming]

I changed the default vlan to something else so I could use VLAN 1 (I have Unifi APs). It partially works (devices get an IP via DHCP) but from that VLAN I cannot reach the internet or other VLANs. Did I miss something I needed to clear in configuring VLAN 1 to make it routable?

Did you create a VE/router-interface in VLAN 1, and assign it the proper address for that subnet?

 

[sbsth]

Did you create a VE/router-interface in VLAN 1, and assign it the proper address for that subnet?

Yup, it has 10.0.1.5/24.

SSH@b24p#show ip route
Total number of IP routes: 6
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.0.255.1      ve 255        1/1           S    2h42m
2       10.0.0.0/24        DIRECT          ve 2          0/0           D    3h0m
3       10.0.1.0/24        DIRECT          ve 1          0/0           D    1h26m
4       10.0.5.0/24        DIRECT          ve 5          0/0           D    3h0m
5       10.0.10.0/24       DIRECT          ve 10         0/0           D    3h0m
6       10.0.255.0/29      DIRECT          ve 255        0/0           D    2h42m

 

[kpfleming]

Yup, it has 10.0.1.5/24.

SSH@b24p#show ip route
Total number of IP routes: 6
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          10.0.255.1      ve 255        1/1           S    2h42m
2       10.0.0.0/24        DIRECT          ve 2          0/0           D    3h0m
3       10.0.1.0/24        DIRECT          ve 1          0/0           D    1h26m
4       10.0.5.0/24        DIRECT          ve 5          0/0           D    3h0m
5       10.0.10.0/24       DIRECT          ve 10         0/0           D    3h0m
6       10.0.255.0/29      DIRECT          ve 255        0/0           D    2h42m

OK, and while this is supposed to be obvious, we've all made obvious mistakes... the DHCP server is handing that address to clients as their default gateway?

 

[sbsth]

OK, and while this is supposed to be obvious, we've all made obvious mistakes... the DHCP server is handing that address to clients as their default gateway?

No offense taken, good to check everything.

It is! I have a Kea DHCP server, for each VLAN subnet it provides the gateway as the switch virtual interface IP (10.0.X.5).

 

[kpfleming]

It is! I have a Kea DHCP server, for each VLAN subnet it provides the gateway as the switch virtual interface IP (10.0.X.5).

Good. Have you also confirmed that the clients on that VLAN can ping the VE IP address?

 

[sbsth]

Good. Have you also confirmed that the clients on that VLAN can ping the VE IP address?

Clients on VLAN 2 can ping the VE address for all other VLANs. They cannot ping the address of clients on any of the other VLANs except VLAN 1, for some reason.

 

[kpfleming]

Clients on VLAN 2 can ping the VE address for all other VLANs. They cannot ping the address of clients on any of the other VLANs except VLAN 1, for some reason.

Do you have any access-lists in your configuration?

 

[sbsth]

Do you have any access-lists in your configuration?

Yes, but only inbound on vlans 5 and 10. I expect to be able to reach VLAN 2 from VLAN 1 or 254.

 

[dbvader]

It partially works (devices get an IP via DHCP) but from that VLAN I cannot reach the internet or other VLANs.

Are clients getting 10.0.255.1 as their gateway address? If so, then I would not expect things to work.

Each client would need to get the ve address of the vlan they're attached to as their gateway address (e.g. 10.0.1.5 for vlan 1) for routing to work.