Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
Thus, question: in simple words, can someone explain the Pros/Cons of setting up VLANs on the pfSense appliance vs setting them up on the 6450 Brocade switch?
I just had a thread on Reddit about this subject, if you want to dig more. Note that @infoMatt is absolutely correct - VLANs are Layer 2 technology, so you'll have to configure on the switch anyhow. What you're reading about being "easier" is routing between VLANs, which is another (though related) subject. Go read the thread, I wrote a bunch of stuff on the topic going back and forth with the original poster. Should help explain the pros and cons.

VLANs for Dummies : homelab
 
  • Like
Reactions: tommybackeast

BobTB

Member
Jul 19, 2019
81
19
8
On the 6430, as it doesn't have routing capabilities, I'd flash the S version (the same as the software it should have now...); a nicer question would be: flashing a router-capable firmware would brick it? Onestly, I don't know, sorry :(
The problem is, that in the zip files on the "how to" site there is no S firmware. I found a firmware on the ruckus website, but it is version 08030r which is older than 08030t. Does aoyone knows where I can get the latest "S" firmware?
 

nerdalertdk

Fleet Admiral
Mar 9, 2017
228
118
43
::1
The ICX7250-48p should have BGP, if i read this right ?

Q. Does the Brocade ICX 7250 have Layer 3 capabilities?
A. All Brocade ICX 7250 models, except the Brocade ICX 7250-24G, have IPv4 and IPv6 Layer 3 capabilities:

• Base Layer 3 routing: Available in Brocade ICX 7250* models, with no license required. It includes IPv4 and IPv6 static routes and routing between directly connected subnets.

• Premium Layer 3 routing: Adds IPv4/IPv6 static and dynamic routes with RIPv1/v2/RIPng announce, VRRP, and OSPFv2/v3. It also includes unicast routing protocols, such as PIM, and rich Layer 3 features, such as Policy-Based Routing (PBR), VRRP, and VRRP-E. Additionally, BGP, VRF capabilities, and IPv4-over-IPv6 tunneling features are included.

• Layer 3 IPv6 and multicast routing: Will be available, with the Premium Layer 3 routing, in a future software release.


Source : https://www.proficomms.cz/files/datasheets/Brocade/brocade-icx-7250-faq.pdf
 

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
that is incredibly odd - I don't know how reflashing firmware would have fixed it (unless it was a different version). The OS image is checksummed entirely at boot so if there was anything corrupt with it, it would refuse to boot (or fall back to secondary OS slot)
when you say remove stack units, do you mean the lines in the config? Remove those, reload switch, and let me know if the behavior returns - that's the only difference between the old setup that was failing and the new one that isn't, as far as I can tell
So I've rolled a bunch of these out in stacks, and I can now confirm that I have a stack which is exhibiting the same behavior. If I tag VLAN's on 1/2/2 or 2/2/2 the whole port goes south. BOTH switches.
 

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
So I've rolled a bunch of these out in stacks, and I can now confirm that I have a stack which is exhibiting the same behavior. If I tag VLAN's on 1/2/2 or 2/2/2 the whole port goes south. BOTH switches.
ICX stacking ports when used to stack are non-configurable. If I had to guess, the fact that you're tagging ports on a stacked port is taken by the switch to mean stacking should be disabled. I would personally think that the opposite would be true (toss an error because can't configure stacked ports) but that's not how they work it.

This doesn't seem related at all to the error issue you're referencing; pretty sure @fohdeesha was right on that one and it was a faulty ASIC/switch.
 

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
The 40G only ports are still stacked, 1/2/2 to 1/2/5 and 1/2/7 to 1/2/10, 2/2/2 to 2/2/5 and 2/2/7 to 2/2/10 are setup as breakouts, unstacked. Like I said, I've done a mess of these, and these are the first ones to show this EXACT behavior. It's very odd.
 

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
Are you getting the ASIC errors too?

Gotta admit, we've seen issues with OS bugs on stacking ports in some of the 08.0.30 code, but I thought that was cleared in the versions most of us are running, and we're seeing it on 7450s at work. Manifests differently, but I wouldn't be surprised if you're seeing an IOS bug that's rare as hell.
 

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
Are you getting the ASIC errors too?

Gotta admit, we've seen issues with OS bugs on stacking ports in some of the 08.0.30 code, but I thought that was cleared in the versions most of us are running, and we're seeing it on 7450s at work. Manifests differently, but I wouldn't be surprised if you're seeing an IOS bug that's rare as hell.
I'm not familiar with how to see the ASIC errors...I don't see them in the logs. Although on bootup, I did see the same QSFP error.
 

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
I'm not familiar with how to see the ASIC errors...I don't see them in the logs. Although on bootup, I did see the same QSFP error.
Yep, the QSFP+ error was what I was referring to. A few replies down from the one you quoted earlier, Fohdeesha mentions that "chow" was the codename for the Marvell ASIC used in the switch.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,729
3,080
113
33
fohdeesha.com
if you're seeing qsfp and/or chow errors on boot there's definitely something defective, the question is how defective. apparently only enough to show up when you mix stack and non stack 40g ports. Don't think the 6610's save boot logs anywhere
 

Churchill

Admiral
Jan 6, 2016
838
213
43
all my qsfp are disabled in an err-disabled stated. Something about licensing? I used the firmware from the beginning of this thread. Am I missing something?

1/2/1 ERR-DIS None None None None No 1 0 cc4e.2417.69bb
1/2/2 ERR-DIS None None None None No 1 0 cc4e.2417.69bc
1/2/3 ERR-DIS None None None None No 1 0 cc4e.2417.69bd
1/2/4 ERR-DIS None None None None No 1 0 cc4e.2417.69be
1/3/1 ERR-DIS None None None None No 1 0 cc4e.2417.69bf
1/3/2 ERR-DIS None None None None No 1 0 cc4e.2417.69c0
1/3/3 ERR-DIS None None None None No 1 0 cc4e.2417.69c1
1/3/4 ERR-DIS None None None None No 1 0 cc4e.2417.69c2
1/3/5 ERR-DIS None None None None No 1 0 cc4e.2417.69c3
1/3/6 ERR-DIS None None None None No 1 0 cc4e.2417.69c4
1/3/7 ERR-DIS None None None None No 1 0 cc4e.2417.69c5
1/3/8 ERR-DIS None None None None No 1 0 cc4e.2417.69c6



40GigabitEthernet1/2/1 is ERR-DISABLED (invalid license), line protocol is down
Port down for 5 days 23 hours 24 minutes 25 seconds
Hardware is 40GigabitEthernet, address is cc4e.2417.69bb (bia cc4e.2417.69bb)
Interface type is unknown
Configured speed 40Gbit, actual unknown, configured duplex fdx, actual unknown
Configured mdi mode AUTO, actual unknown
Member of L2 VLAN ID 1, port is untagged, port state is DISABLED
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled


ICX6650-64 Router(config-if-e40000-1/2/1)#show license
Index License Name Lid License Type Status License Period License Capacity
Stack unit 1:
1 ICX6650-40G-LIC-POD ensHKJFlFeg Normal Active Unlimited 8
2 ICX6650-10G-LIC-POD ensHKJFlFeg Normal Active Unlimited 24
ICX6650-64 Router(config-if-e40000-1/2/1)#
 

am45931472

Member
Feb 26, 2019
87
17
8
So I've read the thread thoroughly, damn its long. Is this the longest thread on STH?

Anyway i'm still looking for a way, anyway, to quiet down the ICX 6610-48P I got. Have done the full software upgrade thanks to the OP who has been amazing on this thread. I currently have this installed with the single 1000w Power supply. I've read many times on here that adding a second power supply can reduce the noise of these switches but I wanted to make sure that that was true of the POE models of these 6610s, not just the non poe models before i go out and get a second one of these 1000w PSUs. The psu my 6610 came with was a revision B.

Also wanted to know if you can run Poe off these switches with the 250w PSUs instead. would 2x 250w psus be quieter than a single 1000 while also allowing for POE? my poe budget is very low, just a few APs

Just for kicks, I removed the top plate of this switch and put a 20' box fan on top of it to cooler it down to see if reducing the temps a ton would spool down the fans more but it doesnt with cpu temp around 45C. other senor temps are between 29-35C

I have a startech openrack, no space limitations. willing to get creative to get this quieter. Home lab
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,729
3,080
113
33
fohdeesha.com
all my qsfp are disabled in an err-disabled stated. Something about licensing? I used the firmware from the beginning of this thread. Am I missing something?

1/2/1 ERR-DIS None None None None No 1 0 cc4e.2417.69bb
1/2/2 ERR-DIS None None None None No 1 0 cc4e.2417.69bc
1/2/3 ERR-DIS None None None None No 1 0 cc4e.2417.69bd
1/2/4 ERR-DIS None None None None No 1 0 cc4e.2417.69be
1/3/1 ERR-DIS None None None None No 1 0 cc4e.2417.69bf
1/3/2 ERR-DIS None None None None No 1 0 cc4e.2417.69c0
1/3/3 ERR-DIS None None None None No 1 0 cc4e.2417.69c1
1/3/4 ERR-DIS None None None None No 1 0 cc4e.2417.69c2
1/3/5 ERR-DIS None None None None No 1 0 cc4e.2417.69c3
1/3/6 ERR-DIS None None None None No 1 0 cc4e.2417.69c4
1/3/7 ERR-DIS None None None None No 1 0 cc4e.2417.69c5
1/3/8 ERR-DIS None None None None No 1 0 cc4e.2417.69c6



40GigabitEthernet1/2/1 is ERR-DISABLED (invalid license), line protocol is down
Port down for 5 days 23 hours 24 minutes 25 seconds
Hardware is 40GigabitEthernet, address is cc4e.2417.69bb (bia cc4e.2417.69bb)
Interface type is unknown
Configured speed 40Gbit, actual unknown, configured duplex fdx, actual unknown
Configured mdi mode AUTO, actual unknown
Member of L2 VLAN ID 1, port is untagged, port state is DISABLED
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled


ICX6650-64 Router(config-if-e40000-1/2/1)#show license
Index License Name Lid License Type Status License Period License Capacity
Stack unit 1:
1 ICX6650-40G-LIC-POD ensHKJFlFeg Normal Active Unlimited 8
2 ICX6650-10G-LIC-POD ensHKJFlFeg Normal Active Unlimited 24
ICX6650-64 Router(config-if-e40000-1/2/1)#

you need licenses to unlock the 40gb ports on the 6650. You have them, but you still need to tell the ports to unlock themselves:

conf t
fpod-40g-enable group 1
fpod-40g-enable group 2
fpod-40g-enable group 3
write mem
 

Churchill

Admiral
Jan 6, 2016
838
213
43
After you enable them with the "fpod" command you need to enable the interfaces manually. I had to go into port 1/2/1 and type "enable" and VOILA! 40gb!