Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
Just with the way the rest of the VLANs work, it’s clunky and disconnected. I ran into some goofy circular issues trying to change the native VLAN on a port to like 9, then tag in some other ones.

It’s just weird to have all your VLANs: you tag your trunk ports, untag your access ports, but then you have to go to a different section to have the untagged VLAN for a trunk port.

In the later Brocade versions that run on the 7250, you just add the port as untagged on the VLAN. It just makes more sense from a configuration standpoint.
Yeah, my major complaint about... well, just about EVERYONE that isn't Cisco last I looked (and maybe Juniper, haven't used their stuff), but Brocade/Ruckus in particular is the way they handle VLANs and tagged vs. untagged.

What actually is nice for anything running 08.0.30 code is that if you set up per-port authentication, you can actually define access port VLANs on the interface - you just use `auth-default-vlan XXX` and as long as the port's authenticated it will work on that VLAN.

I will admit, dual-mode command on the port after you just did `no tagged ethe 1/1/1` for 15 VLANs is really freaking irritating.
 

kroy

New Member
May 17, 2019
24
7
3
Yeah, my major complaint about... well, just about EVERYONE that isn't Cisco last I looked (and maybe Juniper, haven't used their stuff), but Brocade/Ruckus in particular is the way they handle VLANs and tagged vs. untagged.
I definitely hear you there. I’ve been tearing through switches in the last year and have seen much of the good/bad/ugly/etc.

Cisco pretty much simplifies it down on a per interface basis to:
  • Choose native vlan
  • Choose tagged VLANs to allow/deny
  • Choose trunk vs access
The fs.com switches are pretty Cisco-like. They have a nice convenience trick where when you allow a VLAN on an interface, it’s automatically added to the VLAN database. Which is nice for a quick 2-4094.

Arubas go a bit in reverse and make you create profile and then assign that profile to a port. It works well but it’s obnoxious when you have a port that’s a bit of an outlier as you end up creating a profile specifically for that port.

In comparison to those, Brocade definitely isn’t terrible. They go a bit in reverse like the Arubas, and make you assign ports to VLANs instead of VLANs to ports. That would probably be super obnoxious, but the way Brocade accepts port and vlan ranges is definitely helpful.
 

DRAGONKZ

Member
Apr 9, 2018
87
10
8
41
Long time reader, first time poster! (in this thread)

I’m looking at getting 2 x 6610-48Ps and stacking them.

I may have missed posts about this topic, but I just wanted to check that 2 units can still be stacked using the non-breakout 40Gb ports?

I’m ideally wanting to use the 8 x 10Gb front ports, 2 x 40Gb breakout ports as 8 x 10Gb, and still use the 2 x 40Gb non-breakout ports to stack together.

Failing that, can the breakout ports be used for stacking instead?

Thanks!
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Anyone ever have just a single port PoE die suddenly? I rearranged some power cables today, taking the switch down, and now the port I had my AP on no longer comes up. Plugged it into a different one and it works. show inline power says 30000 allocated for a few seconds befoing going back to 0, but the AP doesn't power up.
 

gustav9797

New Member
Dec 30, 2016
19
8
3
26
Checked the power draw of the ICX6450-24P for fun. Looks like it draws 50W at startup and then goes up to a stable 78W after booting, this is with nothing connected but the power cord. A bit far from 21W, is there anything wrong with it?
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
The PoE unit draws a lot more power.
No, it shouldn't... during normal operating conditions, the difference should be negligeable (say 1-2W), other than the load required for the PoE devices themselves, of course.
The "partial load" consumption found on the datasheet is measured as 5% ethernet traffic, full load of PoE PD ;), that's why it's not much different than the 100% load figure.

Personally, I don't have (at the moment?) a power meter, but just by feeling, to me the 75W sounds way off the tracks... I mean, it was pretty hot here in Italy in the last few days, and the exhaust of my ICX 6450-24P felt "by hand" barely warmer than the surrounding environment, even after an entire afternoon in a not-so-well-ventilated room with an ambient temperature of 28-29°C (no aircon of course)... Inline power was enabled on all ports, but there were just two access points connected.
 
Last edited:

pyroguysf

New Member
Aug 17, 2019
3
0
1
I'm looking to upgrade my wireless router + unmanaged switch, and the 6450 seems perfect for future expansion. There's a lot of great resources here, but I had a few questions.
  1. What would I need to set up the 10gb connections to my FreeNAS box and Windows workstation for faster transfers? I was looking at the Intel X520-DA1, but would the Mellanox cards mentioned here work in FreeNAS? It looks like I'd need some transceivers in addtion to specific cables, but I'm having trouble grasping the compatibility. A QSFP breakout seems a little cluttered and unnecessary for my usage.
  2. Would I be able to add an unmanaged switch downstream, or would that cause issues/functionality loss?
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
What would I need to set up the 10gb connections to my FreeNAS box and Windows workstation for faster transfers? I was looking at the Intel X520-DA1, but would the Mellanox cards mentioned here work in FreeNAS? It looks like I'd need some transceivers in addtion to specific cables, but I'm having trouble grasping the compatibility. A QSFP breakout seems a little cluttered and unnecessary for my usage.
Yep, both the Intel and the ConnectX would work, pick the one that you can find at the lowest price.
Other than the NICs you'll need the cabes, and you have basically two choices:
  • SFP+ DAC (direct attach cable), if you need to cover a short distance and they don't need to be routed through the walls/conduits, or
  • SFP+ optics and fiber optic: these switches don't care about the brand of the optics, but if it's not Brocade you wouldn't see the monitoring details (ie. optical power received and temperature, among the others...). If you choose this route you'll have to match fiber types and optics at both ends (single mode vs multimode, for in-house purpose basically consider the shortest range optics, ie. SR for MM and LR for single mode). Don't quote me on that, but from my memory the Intel NICs by default are picky about the optics, and require Intel ones, but I've not tried myself (this is what I've been told at $dayjob, but can't confirm or deny).
EDIT: QSFP breakouts are used at the switch side for the 40G ports, and there are none on the 6450; if you find a 40G QSFP NIC you'll need a QSFP-to-SFP+ converter (look at the last few pages).

Would I be able to add an unmanaged switch downstream, or would that cause issues/functionality loss?
No problems; watch for any loops in the network, because if you make a loop ad activate the spanning tree, the switch might ERR-DISABLE the downlink port ;).
 
Last edited:

pyroguysf

New Member
Aug 17, 2019
3
0
1
Thanks Matt.
  • SFP+ DAC (direct attach cable), if you need to cover a short distance and they don't need to be routed through the walls/conduits, or
  • SFP+ optics and fiber optic: these switches don't care about the brand of the optics
So you don't need optics with a DAC? I guess the direct attach part makes sense now. When I was looking at FS, their DACs had a different option for each manufacturer - does that matter if optics aren't involved, or is it just more of a marketing thing? Should I just get one that matches the NIC, or would I need a dual compatible one?

Similarly, I read about Intel NICs being optics locked, so if I went that route, I'd want an Intel transceiver at each end with an LC-LC OM1 cable? If I went Mellanox, would the Brocade transceiver work on that end for the extra details?

QSFP breakouts are used at the switch side for the 40G ports, and there are none on the 6450; if you find a 40G QSFP NIC you'll need a QSFP-to-SFP+ converter (look at the last few pages).
That makes much more sense, I don't know how I missed that. I swear I was reading it as people using breakouts on the NIC side which made no sense.

Edit: I've read conflicting reports - if I connect my NAS and workstation to the SFP+, will I be able to transfer at 10Gb?
 
Last edited:

French Chamallow

New Member
Mar 15, 2019
17
3
3
Checked the power draw of the ICX6450-24P for fun. Looks like it draws 50W at startup and then goes up to a stable 78W after booting, this is with nothing connected but the power cord. A bit far from 21W, is there anything wrong with it?
can other users with a powermeter confirm that the ICX6450-24P consumes so much watt in idle ?
 

klui

Well-Known Member
Feb 3, 2019
824
453
63
Not according to the main post and the specifications here: Ruckus ICX 6450-24P
:rolleyes:
Check your meter's other measurements. I have a brand new ICX7150-C12P and my Kill-a-Watt measures 36W with only the console cable connected.

This Kill-a-Watt is used intermittently, but I have shut off the circuit while it was plugged in before. Its LCD is currently slow to update and you can see the LCD segments' refresh rate is slow even when values aren't changing. The voltage reading is on the outlet with nothing plugged in is 180V and that isn't right. It displays that voltage for every other single-phase circuit in my house. A CatII multimeter shows normal voltage at 120V. So the measurement for wattage will be off due to voltage, at least. If I convert using 120 instead of 180, I get roughly 24W or so. Shutting off PoE on all ports don't change the wattage value from the Kill-a-Watt.
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
Thanks Matt.

So you don't need optics with a DAC? I guess the direct attach part makes sense now. When I was looking at FS, their DACs had a different option for each manufacturer - does that matter if optics aren't involved, or is it just more of a marketing thing? Should I just get one that matches the NIC, or would I need a dual compatible one?

Similarly, I read about Intel NICs being optics locked, so if I went that route, I'd want an Intel transceiver at each end with an LC-LC OM1 cable? If I went Mellanox, would the Brocade transceiver work on that end for the extra details?
Exactly, the scope of a DAC cable is to avoid the cost of the dual optic when the distance is short enough...
The vendor "signature" of the DAC works in the same way as the optics, it is a way for the vendor to recognize that you're using a chain of "verified components"; if a switch or a NIC wouldn't work with a generic/other vendor signed optics, then it wouldn't bring up the link if you use a not-verified DAC (I've tried this with a Cisco original SFP+ DAC in a HP Procurve 2810, the switch refused to link and it shows as a tampered with optics).

In your case, you could buy an Intel-Intel DAC or an Intel-Brocade one, for your switch it doesn't make any difference and for the DACs you don't get any monitoring anyway, so... no big deal ;)

If you go to the dual optic + fiber route, you'll need a Brocade SFP+ 10G-SR, an Intel SFP+ 10G-SR and a OM3 or OM4 patch cable.. the OM1 has a bigger core of 62.5nm (instead of the 50nm of the "modern" OM3+ multimode fibers). You might need to "roll the fiber" (ie. swap the fibers left-to-right inside the LC bundle) at one end, in order to connect the TX of one side to the RX of the other... it's more difficult to write it as to do it, and once you've done it once, it would be "second nature"... And, if you use the 850nm wavelength SR optics, 99% of the time you can actually see the light emitted, so you wouldn't do it wrong ;)

Edit: I've read conflicting reports - if I connect my NAS and workstation to the SFP+, will I be able to transfer at 10Gb?
Well.. it depends ;) for short burst then yes, the link would be at 10G so until you have data buffered in memory then yes, you'll get around 1.2GB/s... after you've depleted the buffers, than you're obviously limited by the slowest device in the chain, and most of the time is a spinning disk, so you'll hovering around the 150-180MB/s mark (for bigger files, if you're going to trasfer a lot of small files the performace will obviously drop). ;)

I hope that what I've written makes sense...
 
  • Like
Reactions: kiteboarder

pyroguysf

New Member
Aug 17, 2019
3
0
1
That's a ton of info on the connections, thanks Matt!

I guess I phrased that bad, but I know I won't get 10Gb, I'm just tired of the gigabit connection being my bottleneck. I just read some places that it won't pass through 10Gb to 10Gb on one connection, only saturate multiple gigabit connections from a 10Gb.

A few 10Gb and 24 gigabits will keep me set for a while, so I think I'm going to pull the trigger.
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
That's a ton of info on the connections, thanks Matt!

I guess I phrased that bad, but I know I won't get 10Gb, I'm just tired of the gigabit connection being my bottleneck. I just read some places that it won't pass through 10Gb to 10Gb on one connection, only saturate multiple gigabit connections from a 10Gb.

A few 10Gb and 24 gigabits will keep me set for a while, so I think I'm going to pull the trigger.
Yes, there might be some tweaking of the parameters, especially if you're using a *nix based NAS, but it would be a problem only if you have a ton of HDD in a RAID0 or 5 configuration (or a full solid state archival solution), because the combined bandwidth of the drive could be enough (if the disk controller, CPU and OS allows for it) to saturate a multigig link. If your NAS has a RAID1 configuration, you're limited obviously by the write speed of the slowest drive, say 150-200MB/s, or ~2500Mbps.
Try looking for some tweaks (if any) regarding your NAS and your workstation OS version, if the performance doesn't sound right for you. ;)

There could be some more tweaks to do if you go to the LACP route (ie. multiple interfaces bonded to share their bandwidth), so if the packet hashing algorithm isn't perfect, there could be some cases where a single connection in a bond is forwarded on just one of the links, effectively limiting its speed; if that's the case, using a multi-connection SMB will avoid the problem and share more efficiently the links available.
 

dwright1542

Active Member
Dec 26, 2015
377
73
28
50
Is there any functional reason to use dual stacking cables between switches, rather than just a single cable, when I'm running in a ring topology? Is it for bandwidth reasons?