Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

tommybackeast

Active Member
Jun 10, 2018
232
73
28
a basic VLAN question : I have a ICX 6450-24p and have just been using it for a 10GB Switch, while using none of other features. I had planned to learn VLANS, more networking this summer; but Life occurred.

I currently use a simple Asus-Merlin consumer router/AP.

I am now looking to get a 1U appliance to run pfSense on.

Please note I am a network noob - I had thought that since the 6450 is a Layer 3 Switch; that I should make the VLANs on the Switch; but am now reading it is "cleaner" and easier for a network rookie to make VLANs on the pfSense appliance.

Thus, question: in simple words, can someone explain the Pros/Cons of setting up VLANs on the pfSense appliance vs setting them up on the 6450 Brocade switch?
 

BobTB

New Member
Jul 19, 2019
16
3
3
In the guide for flashing the firmware ther is ICX6430 / ICX6450 section, since I have two ICX6430, can I actually flash the ICX64R08030t.bin to it? Is there also ICX64S08030t.bin ?
 

infoMatt

Active Member
Apr 16, 2019
148
53
28
a basic VLAN question : I have a ICX 6450-24p and have just been using it for a 10GB Switch, while using none of other features. I had planned to learn VLANS, more networking this summer; but Life occurred.

I currently use a simple Asus-Merlin consumer router/AP.

I am now looking to get a 1U appliance to run pfSense on.

Please note I am a network noob - I had thought that since the 6450 is a Layer 3 Switch; that I should make the VLANs on the Switch; but am now reading it is "cleaner" and easier for a network rookie to make VLANs on the pfSense appliance.

Thus, question: in simple words, can someone explain the Pros/Cons of setting up VLANs on the pfSense appliance vs setting them up on the 6450 Brocade switch?
No advantage or disadvantage: if you want L2 isolation, you have to define VLANs on both pfSense and the switches, otherwise the clients will all be on the same broadcast domain.

VLAN are layer 2 topology... i.e. even a plain managed layer2 switch can understand and manage VLAN; the advantage of a L3 switch is that it can route (at line speed) between vlan, instead of having to loop the traffic to the router/firewall and back to the switch again ("router on a stick") tagged with another VLAN.
Instructions of how to define VLAN on pfSense is out of this topic, you can start by watching some video tutorials, it not difficult.

If you want to use the switch in L3 mode, you have to define a virtual interface on each VLAN, each one with it's own IP address, and use it as the default gateway for the client in that network; to avoid having full routing visibility you'll have to dig through ACLs; there are some examples in this thread.

In the guide for flashing the firmware ther is ICX6430 / ICX6450 section, since I have two ICX6430, can I actually flash the ICX64R08030t.bin to it? Is there also ICX64S08030t.bin ?
"R" firmware does support the L3/routing features, the "S" one is just for layer 2 (ie. VLAN)... there's no advantage of using the S version on the 6450, apart for a slight reduction on image size and maybe a couple of seconds at boot...
On the 6430, as it doesn't have routing capabilities, I'd flash the S version (the same as the software it should have now...); a nicer question would be: flashing a router-capable firmware would brick it? Onestly, I don't know, sorry :(
 
Last edited:
  • Like
Reactions: tommybackeast

tommybackeast

Active Member
Jun 10, 2018
232
73
28
[QUOTE="; the advantage of a L3 switch is that it can route (at line speed) between vlan, instead of having to loop the traffic to the router/firewall and back to the switch again ("router on a stick") tagged with another VLAN. :([/QUOTE]


That makes sense, thanks
 

Wolfstar

Active Member
Nov 28, 2015
155
79
28
45
Thus, question: in simple words, can someone explain the Pros/Cons of setting up VLANs on the pfSense appliance vs setting them up on the 6450 Brocade switch?
I just had a thread on Reddit about this subject, if you want to dig more. Note that @infoMatt is absolutely correct - VLANs are Layer 2 technology, so you'll have to configure on the switch anyhow. What you're reading about being "easier" is routing between VLANs, which is another (though related) subject. Go read the thread, I wrote a bunch of stuff on the topic going back and forth with the original poster. Should help explain the pros and cons.

VLANs for Dummies : homelab
 
  • Like
Reactions: tommybackeast

BobTB

New Member
Jul 19, 2019
16
3
3
On the 6430, as it doesn't have routing capabilities, I'd flash the S version (the same as the software it should have now...); a nicer question would be: flashing a router-capable firmware would brick it? Onestly, I don't know, sorry :(
The problem is, that in the zip files on the "how to" site there is no S firmware. I found a firmware on the ruckus website, but it is version 08030r which is older than 08030t. Does aoyone knows where I can get the latest "S" firmware?
 

nerdalertdk

Fleet Admiral
Mar 9, 2017
151
63
28
::1
The ICX7250-48p should have BGP, if i read this right ?

Q. Does the Brocade ICX 7250 have Layer 3 capabilities?
A. All Brocade ICX 7250 models, except the Brocade ICX 7250-24G, have IPv4 and IPv6 Layer 3 capabilities:

• Base Layer 3 routing: Available in Brocade ICX 7250* models, with no license required. It includes IPv4 and IPv6 static routes and routing between directly connected subnets.

• Premium Layer 3 routing: Adds IPv4/IPv6 static and dynamic routes with RIPv1/v2/RIPng announce, VRRP, and OSPFv2/v3. It also includes unicast routing protocols, such as PIM, and rich Layer 3 features, such as Policy-Based Routing (PBR), VRRP, and VRRP-E. Additionally, BGP, VRF capabilities, and IPv4-over-IPv6 tunneling features are included.

• Layer 3 IPv6 and multicast routing: Will be available, with the Premium Layer 3 routing, in a future software release.


Source : https://www.proficomms.cz/files/datasheets/Brocade/brocade-icx-7250-faq.pdf
 

dwright1542

Active Member
Dec 26, 2015
362
69
28
47
that is incredibly odd - I don't know how reflashing firmware would have fixed it (unless it was a different version). The OS image is checksummed entirely at boot so if there was anything corrupt with it, it would refuse to boot (or fall back to secondary OS slot)
when you say remove stack units, do you mean the lines in the config? Remove those, reload switch, and let me know if the behavior returns - that's the only difference between the old setup that was failing and the new one that isn't, as far as I can tell
So I've rolled a bunch of these out in stacks, and I can now confirm that I have a stack which is exhibiting the same behavior. If I tag VLAN's on 1/2/2 or 2/2/2 the whole port goes south. BOTH switches.
 

Wolfstar

Active Member
Nov 28, 2015
155
79
28
45
So I've rolled a bunch of these out in stacks, and I can now confirm that I have a stack which is exhibiting the same behavior. If I tag VLAN's on 1/2/2 or 2/2/2 the whole port goes south. BOTH switches.
ICX stacking ports when used to stack are non-configurable. If I had to guess, the fact that you're tagging ports on a stacked port is taken by the switch to mean stacking should be disabled. I would personally think that the opposite would be true (toss an error because can't configure stacked ports) but that's not how they work it.

This doesn't seem related at all to the error issue you're referencing; pretty sure @fohdeesha was right on that one and it was a faulty ASIC/switch.
 

dwright1542

Active Member
Dec 26, 2015
362
69
28
47
The 40G only ports are still stacked, 1/2/2 to 1/2/5 and 1/2/7 to 1/2/10, 2/2/2 to 2/2/5 and 2/2/7 to 2/2/10 are setup as breakouts, unstacked. Like I said, I've done a mess of these, and these are the first ones to show this EXACT behavior. It's very odd.
 

Wolfstar

Active Member
Nov 28, 2015
155
79
28
45
Are you getting the ASIC errors too?

Gotta admit, we've seen issues with OS bugs on stacking ports in some of the 08.0.30 code, but I thought that was cleared in the versions most of us are running, and we're seeing it on 7450s at work. Manifests differently, but I wouldn't be surprised if you're seeing an IOS bug that's rare as hell.
 

dwright1542

Active Member
Dec 26, 2015
362
69
28
47
Are you getting the ASIC errors too?

Gotta admit, we've seen issues with OS bugs on stacking ports in some of the 08.0.30 code, but I thought that was cleared in the versions most of us are running, and we're seeing it on 7450s at work. Manifests differently, but I wouldn't be surprised if you're seeing an IOS bug that's rare as hell.
I'm not familiar with how to see the ASIC errors...I don't see them in the logs. Although on bootup, I did see the same QSFP error.
 

Wolfstar

Active Member
Nov 28, 2015
155
79
28
45
I'm not familiar with how to see the ASIC errors...I don't see them in the logs. Although on bootup, I did see the same QSFP error.
Yep, the QSFP+ error was what I was referring to. A few replies down from the one you quoted earlier, Fohdeesha mentions that "chow" was the codename for the Marvell ASIC used in the switch.
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,857
1,660
113
29
fohdeesha.com
if you're seeing qsfp and/or chow errors on boot there's definitely something defective, the question is how defective. apparently only enough to show up when you mix stack and non stack 40g ports. Don't think the 6610's save boot logs anywhere