Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

tommybackeast

Active Member
Jun 10, 2018
258
85
28
All the bugs in that list were closed/fixed in this release so you'll get them, "found in" lists all the builds that the bug was present in (08.0.95 being a currently unreleased internal test train)
thanks for the info, but I don't change brocade firmware till you say its ok to update.
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
I have a very basic question for you experienced people...

it's taken me a damn year but I finally am brave enough to switch over to a pfSense dedicated box; and making VLANs. A friend and I both picked up Brocade 7250, a Ruckus r510 AP; and a dedicated PC for pfsense. Home network.

1. if VLANs are created on the pfsense box, does the layer 3 VLAN communication also occur on the pfsense box or can you create VLANs on pfSense box and then Layer3 them on the Brocade 7250 ?

2. is it easier/smarter/more-secure making VLANs on the Brocade 7250?and just let the pfsense box act as router with some packages running

3. I am right now starting at the Brocade 7250 WebGUI and see the VLAN section (don't understand it, but i see it). Can VLANs be made here or best practice to only use CLI (for a total network noob please note)

I have all the hardware : pfsense PC, Brocade 7250 with console cable, Ruckus r510 AP, two Synology NAS with 1GB & 10GB NIC ports, Dell Server with 1GB and 10GB NIC ports - ESXi - lots of VMs

Simply do not understand if VLANs get made on pfsense or Brocade; dont understand if the VLAN Layer3 communication takes place on the pfsense box or Brocade.

My friend was going to ask reddit today;' but I thought I would ask the brocade experts here....

I apologize in advance to all the IT Pros reading this :( for the basic nature of the question
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
about to head out but wanted to say thank you for the URL which I shall read later tonight.

from the URL you gave me, it starts with "With a layer 3 switch, the general recommendation is to either let the switch do DHCP duties (it can), or have a dedicated DHCP server that can handle VLANs (pfSense cannot, at this time...)."

My head is spinning from all the reading I've done the last few days; but am I nuts - doesn't pfSense current version 2.4.4 offer DHCP Server built in ?
 

klui

Active Member
Feb 3, 2019
194
82
28
"With a layer 3 switch, the general recommendation is to either let the switch do DHCP duties (it can), or have a dedicated DHCP server that can handle VLANs (pfSense cannot, at this time...)."

My head is spinning from all the reading I've done the last few days; but am I nuts - doesn't pfSense current version 2.4.4 offer DHCP Server built in ?
In order for a switch to perform L3 duties it has to have an interface for your subnet that acts as a gateway. Then the switch can perform ACLs and perform routing to other subnets. Yeah, pfSense has DHCP functions but most how-tos for the configuration will set the pfSense interface as the gateway and you need to either change that or define a DHCP server on an L3 switch and use that and add an additional route to pfSense for your upstream gateway.
 
  • Like
Reactions: tommybackeast

tommybackeast

Active Member
Jun 10, 2018
258
85
28
In order for a switch to perform L3 duties it has to have an interface for your subnet that acts as a gateway. Then the switch can perform ACLs and perform routing to other subnets. Yeah, pfSense has DHCP functions but most how-tos for the configuration will set the pfSense interface as the gateway and you need to either change that or define a DHCP server on an L3 switch and use that and add an additional route to pfSense for your upstream gateway.
first thank you for replying. Given your language, you have knowledge and experience.

Bluntly, my own lack of knowledge prevents me from deeply understanding your above comments. (sorry)

I ask my question as a newbie to all of this who is truly trying to learn but also maintain good security on the home network (but lacks knowledge when stepping away from the asus router/AP and 192.168.1.0/24 I have used for a long time.

If I am understanding you correctly, current version pfSense does have DHCP Server but does it in a poor manner? is that accurate?

How much "harder" (for a newbie) is letting the pfsense router just be a router with some packages like suricata , pfblockerng -and- DHCP Server, VLANS + L3 being done on the Brocade 7250?

I suspect you do everything CLI on the brocade but might you know how 'easy' the brocade WebGUI is to setup VLANS + L3 ?
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
In order for a switch to perform L3 duties it has to have an interface for your subnet that acts as a gateway. Then the switch can perform ACLs and perform routing to other subnets. Yeah, pfSense has DHCP functions but most how-tos for the configuration will set the pfSense interface as the gateway and you need to either change that or define a DHCP server on an L3 switch and use that and add an additional route to pfSense for your upstream gateway.
Allow me to ask a sideways question: I now will have a 10GBe Network : Brocade 7250, 2 Synology NAS with 10GBe NIC, Dell Server / ESXi with 10GB ; and two PC computers with 10GB NICs.

Pfsense router has two 1GB NICs

Nothing is really set up yet. If I run pfSense with DHCP Server, VLANs and L3 switching : does this mean all the LAN traffic will pass through the pfsense 1GB NICs?

Say I move a 5GB file from Synology NAS (on STORAGE-VLAN) to PC Computer [on LAN-VLAN] (10GB in both boxes). Will that 5GB file LAN transfer pass through the 1GB NIC in the pfsense router? (thus making have 10GB NICs useless) ? (note above two VLAN names are just made up for the example, still in process of trying to figure out how to start this process)
 

klui

Active Member
Feb 3, 2019
194
82
28
Allow me to ask a sideways question: I now will have a 10GBe Network : Brocade 7250, 2 Synology NAS with 10GBe NIC, Dell Server / ESXi with 10GB ; and two PC computers with 10GB NICs.

Pfsense router has two 1GB NICs

Nothing is really set up yet. If I run pfSense with DHCP Server, VLANs and L3 switching : does this mean all the LAN traffic will pass through the pfsense 1GB NICs?
Yes, that is one of the reasons why you want to use an L3 switch. Because typically your DHCP server will have a scope and the gateway will point to the interface where the DHCP server is listening to. The other benefit of using an L3 switch like the ones discussed in this thread is rules are performed in hardware so you will get line rate (1G, 10G, 40G) performance. Under pfSense installed in white-box hardware, your firewall rules will most likely be done in software so you will need a robust CPU to achieve line rate.

Having your pfSense firewall as the gateway won't make your 10G NICs useless. But they will need to have access to the pfSense interface if the destination isn't already known by the switch. If it is known by the switch then those transactions will be routed properly; however, if you have FW rules then your packets must go back to pfSense first.

pfSense doesn't have a bad implementation but you need to look at what HW you have and what kind of performance you require.
 
  • Like
Reactions: tommybackeast

eduncan911

Active Member
Jul 27, 2015
100
44
28
65
eduncan911.com
Wanted to post an update on my infinite quest to silence an ICX 7250-48P (I run this thing in a fairly warm closet with bad circulation). Call me obsessive, but I wasn't a fan (no pun intended) of my temps creeping into the low 70s after my previous fan swap. I believe I have come up with the perfect silent solution for this rig.

Replace all 3 rear fans with the Mechatronics MR4020X12B1-RSR fan. They will read just fine in CLI. Then add a Sunon MF60101V3-1000U-A99 fan to the ASIC heat sink (you'll still have around 1/3" to spare in the case, plenty). Wire it in with Fan 3. All the fans are reduced load vs stock so they won't draw too much and the Sunon only has power and ground, so doesn't affect the read on Fan 3.

The switch now runs next-to-dead silent; temperatures on the ASIC dropped ~20 degrees over my previous configuration into the low to mid 50s degrees and the PSU is in the upper 20s which is great. Been running around a week, everything has been perfect. I'm in love.
Could you post some pics?

Also, where did you source these from? I found the Mechatronics only at digikey and the Sunon at Mouser - not both at one. Then again, I do have some parts I have to get some Mouser anyways... :)
 

Fallen Kell

New Member
Mar 10, 2020
29
7
3
Sorry for asking something that might have already been answered. I just picked up a 6610-24 (non-PoE). I have seen others asking about quieting the 6610 before and it was stated not to try. Now I can understand that for the PoE versions (which have ~1000W power supplies) but I find it surprising that a switch which nominally draws 120W can not be cooled quietly. Are there really no methods to quiet this switch with modifications?

It is by far the loudest item in my rack (even over my supermicro cse-846).
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,003
1,824
113
29
fohdeesha.com
Sorry for asking something that might have already been answered. I just picked up a 6610-24 (non-PoE). I have seen others asking about quieting the 6610 before and it was stated not to try. Now I can understand that for the PoE versions (which have ~1000W power supplies) but I find it surprising that a switch which nominally draws 120W can not be cooled quietly. Are there really no methods to quiet this switch with modifications?

It is by far the loudest item in my rack (even over my supermicro cse-846).
no, and it shouldn't be that loud, I can't hear mine over an R720. do the fan spins way down after boot, and what revision are the power supplies?
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
no, and it shouldn't be that loud, I can't hear mine over an R720. do the fan spins way down after boot, and what revision are the power supplies?
The loudest thing in my XrackPro2 enclosed cabinet are the rear XrackPro2 fans themselves lol.

I"m not home; and forget if its 3 80mm or 120mm fans, but can anyone suggest a Mfg of DC fans that are quiet but still move air?
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
Yes, that is one of the reasons why you want to use an L3 switch. Because typically your DHCP server will have a scope and the gateway will point to the interface where the DHCP server is listening to. The other benefit of using an L3 switch like the ones discussed in this thread is rules are performed in hardware so you will get line rate (1G, 10G, 40G) performance. Under pfSense installed in white-box hardware, your firewall rules will most likely be done in software so you will need a robust CPU to achieve line rate.

Having your pfSense firewall as the gateway won't make your 10G NICs useless. But they will need to have access to the pfSense interface if the destination isn't already known by the switch. If it is known by the switch then those transactions will be routed properly; however, if you have FW rules then your packets must go back to pfSense first.

pfSense doesn't have a bad implementation but you need to look at what HW you have and what kind of performance you require.
Thank you for explaining : I got a bit confused from reading /r/homelab; where lots of guys mix PROD and LAB; and got used to seeing them talk about a large of VLANs.

so I know understand why (for me, a newbie) keep all the 10GB devices in the same, primary VLAN as my two Workstation computers.

until 15min ago, I was planning on DHCP server on Brocade but just read foodeesha doesn't really suggest doing that, given the understandable comment that Brocade's real enterprise clients are doing DHCP Server on a different dedicated box; so their implementation is not perfect. (of course, I have just spent an hour reading the Brocade manual on DHCP Server before visiting STH.com, lol

Question: is your own DHCP Server on your Brocade? any problems?
 

tommybackeast

Active Member
Jun 10, 2018
258
85
28
Wanted to post an update on my infinite quest to silence an ICX 7250-48P (I run this thing in a fairly warm closet with bad circulation). Call me obsessive, but I wasn't a fan (no pun intended) of my temps creeping into the low 70s after my previous fan swap. I believe I have come up with the perfect silent solution for this rig.

Replace all 3 rear fans with the Mechatronics MR4020X12B1-RSR fan. They will read just fine in CLI. Then add a Sunon MF60101V3-1000U-A99 fan to the ASIC heat sink (you'll still have around 1/3" to spare in the case, plenty). Wire it in with Fan 3. All the fans are reduced load vs stock so they won't draw too much and the Sunon only has power and ground, so doesn't affect the read on Fan 3.

The switch now runs next-to-dead silent; temperatures on the ASIC dropped ~20 degrees over my previous configuration into the low to mid 50s degrees and the PSU is in the upper 20s which is great. Been running around a week, everything has been perfect. I'm in love.
now please teach me how to quiet the 3 fans on my 12U XrackPro2 server cabinet please, lol

My own 7250 is ok for me, but compliments to your work.
 
  • Like
Reactions: acpatel

snclawson

Member
Feb 7, 2013
46
18
8
So it seems that I've found yet another rabbit hole to fall into care of STH!

This time my home network is getting a workover. I've already got the pfSense box. A pair of R500 access points are showing up today and I bought a cheap Mokerlink PoE unmanaged switch before I ran into this thread. As it turns out, I've also been moving a bunch of data between my main PC and my NAS recently and the single 1G link that it's currently connected to the network with is becomming a real annoyance.

Sooo...a cheap(ish), quiet box that does PoE and has at least two 10G SFP+ ports would be perfect.

Other than it being a little more than I was hoping to spend, the ICX7150-C12P seems to be it. =) But the ICX6450 is intriguing, especially if it comes around at $100 + shipping often enough, since it's got 2 extra 10G ports.


Two quick questions though, since I haven't be able to read the entire thread yet and either I'm incompetent with`search' or it's not mentioned:

- On the switches with Broadcom ASICs (ICX7150/ICX7250), is `bshell' available? Maybe through the console/debug terminal?

- I saw mention of what Marvell chip was in the ICX6450 as the CPU (although I can't find that again either!), but what switching ASIC is it using? I've seen mention of Prestera for the FCX line, but not the ICX6xxx?
 
  • Like
Reactions: tommybackeast

fohdeesha

Kaini Industries
Nov 20, 2016
2,003
1,824
113
29
fohdeesha.com
So it seems that I've found yet another rabbit hole to fall into care of STH!

This time my home network is getting a workover. I've already got the pfSense box. A pair of R500 access points are showing up today and I bought a cheap Mokerlink PoE unmanaged switch before I ran into this thread. As it turns out, I've also been moving a bunch of data between my main PC and my NAS recently and the single 1G link that it's currently connected to the network with is becomming a real annoyance.

Sooo...a cheap(ish), quiet box that does PoE and has at least two 10G SFP+ ports would be perfect.

Other than it being a little more than I was hoping to spend, the ICX7150-C12P seems to be it. =) But the ICX6450 is intriguing, especially if it comes around at $100 + shipping often enough, since it's got 2 extra 10G ports.


Two quick questions though, since I haven't be able to read the entire thread yet and either I'm incompetent with`search' or it's not mentioned:

- On the switches with Broadcom ASICs (ICX7150/ICX7250), is `bshell' available? Maybe through the console/debug terminal?

- I saw mention of what Marvell chip was in the ICX6450 as the CPU (although I can't find that again either!), but what switching ASIC is it using? I've seen mention of Prestera for the FCX line, but not the ICX6xxx?
all icx6xxx series are marvell, icx7xxx is broadcom. If I recall correctly up-n-atom and I found the hidden debug shell, it required a bunch of u-boot args to get the system to boot in a state where it was allowed. What are you trying to do that's not exposed through the regular cli?
 
  • Like
Reactions: tommybackeast