Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

ljvb

Member
Nov 8, 2015
97
32
18
46
I agree, they work perfectly fine, however Windows has much higher overhead. No sense in wasting resources if you don't have to. Just another option :) thats all.
 

kapone

Well-Known Member
May 23, 2015
1,056
627
113
Not disagreeing, but sometimes you choose Windows because you're doing other things as well, and/or you're a Windows shop, and/or you like GUIs... :)
 

ljvb

Member
Nov 8, 2015
97
32
18
46
Guis are the devils work........

I'm just happy I have things figured out.
Have the 2 10G NC523SFP links up and running as a lag connected to the 6610 with the breakout cable. I dropped 40G optics into the the 2 switches for the uplink. I have the 50M cable, but I still need to switch the polarity.

Not going to bother with a 10G card for the pfsense box, since I am no longer doing vlan control through it, and I only have 200Mbit symetric fiber (FIOS), I don't see the need.

The one issue I had, so far just happened once, and I am not sure if it's related to thermal protection or not.. the links of the 10G card went dead, no carrier. I pulled the module on the switch side and reseated it, no luck, I pulled the transceiver from the cards ont eh VM Server, reseated, no luck, eventually I just rebooted the switch, and they came back. I have not tried rebooting the vmserver as it takes 10 minutes to bootup and start the virtual machines, with some custom scripting because the iscsi mounts are mounted from freenas which is a VM itself (passthrough for 2 m1015 hbas).
 
  • Like
Reactions: fohdeesha

PGlover

Active Member
Nov 8, 2014
498
63
28
55
Gave the built in DHCP server a go to see if I would run into your issue, and I did indeed with a couple IOT type stuff. After a ton of debugging, it turned out to be the fact the FastIron DHCP server is not set as authoritative, and some DHCP stack implementations (like Roku's and Sonos) do not like this, and will ignore it.

I went ahead and just spun up a really lightweight debian VM running isc-dhcp-server, it has it's own transit vlan to the switch, and the switch is set to relay all DHCP requests to it.

Gives me WAY more control than the built in dhcp server in FastIron or pfsense, and with glass-isc-dhcp you get really nice visualizations and lookup of all leases that you wouldn't get otherwise: Akkadius/glass-isc-dhcp

let me know if you want some help setting it up as well as my isc-dhcp config, it's as simple as just apt install isc-dhcp-server then copy in a config, and it'll serve DHCP requests to multiple routed vlans on your switch (just need an ip-helper statement under each VE on the switch)
You are awesome.. I had an existing WIN 2016 Server VM and installed the DHCP service. Created the IP helper on my ICX6610 and I am now able to get Sonos working on the switch. You just made my weekend. Thank you so much for all your knowledge and willingness to help everyone. Kudos to you.
 

Juan C

New Member
Oct 9, 2018
8
7
3
Greensboro, NC
Firmware + Docs:
NOTE: If you buy one of these switches, use the update guide at the following link. It will get you initially set up with a fresh slate, the latest firmware, latest documentation, and run you through the basic setup (choose the appropriate switch article on the left): FCX / ICX6610 - FBOM Docs
For anyone trying to figure out why the web interface won't load anything but the port map, be sure to take a look at the getting started guide! I lost quite a bit of time because I didn't flash the newest firmware from the get-go.
 
  • Like
Reactions: Emanuele

nezach

Active Member
Oct 14, 2012
208
118
43
I have multiple VLANs in my home network LAN(Wired)/Wifi/IOT/DMZ etc. and primary reason for separating hosts into separate VLANs is to isolate them from each other for security reasons. I use firewall in pfSense to allow/restrict traffic between VLANs as well as to/from outside.

When you guys use L3 switch to do inter-VLAN routing do you utilize ACLs to filter traffic flow or do you allow everything. I am trying to better understand when it is beneficial to switch to using L3 switch for that purpose instead of a firewall. Thanks!
 

ljvb

Member
Nov 8, 2015
97
32
18
46
I have multiple VLANs in my home network LAN(Wired)/Wifi/IOT/DMZ etc. and primary reason for separating hosts into separate VLANs is to isolate them from each other for security reasons. I use firewall in pfSense to allow/restrict traffic between VLANs as well as to/from outside.

When you guys use L3 switch to do inter-VLAN routing do you utilize ACLs to filter traffic flow or do you allow everything. I am trying to better understand when it is beneficial to switch to using L3 switch for that purpose instead of a firewall. Thanks!
If I was at work, or at a client site, ACLs, lots of ACLs. The default policy between networks should be default deny permit by exception...

That said, at home. I am lazy, I permit all traffic between networks.. I will be adding a security camera vlan (rash of car break ins and people stealing off peoples docks on the water side), which I will isolate. I might do a more appropriate secure configuration as I renovate my house and put drops into each room and redo everything.
 

ljvb

Member
Nov 8, 2015
97
32
18
46
@kapone @fohdeesha
Does LLDP/CDP not work on the 40GB to 10GB with breakout cables? I have no issues with it on the gig ports, but after I put in the dualport 10G NC523 into my VM server replacing the quadport gig intel car, I am no longer getting the data from the switch on the distributed vmware switch.

As far as I can tell, I have it enabled (I did issue lldp enable ports all).
 

Juan C

New Member
Oct 9, 2018
8
7
3
Greensboro, NC
On a complete side-note, anyone know whether it's possible to configure a ConnectX-2 VPI card to connect to the Brocade via the 40Gbps connection and be configured in >10Gbps mode?

<EDIT>
I just figured it out. The ConnectX-2 VPI card only supports 10Gb over Ethernet and 40Gb over IB. The ConnectX-3 VPI card supports 40Gb over Ethernet. Well, ebay, here I come!
 
Last edited:

tommybackeast

Active Member
Jun 10, 2018
290
111
43
Odd question: re Brocade 6450-24p. I am just now filling my first-ever Server Rack Cabinet (12U, xRackPro2) which has front and rear doors (and side doors). The Brocade 6450 is top of rack, Rear.

Cable management question: is it 'safe' to feed Cat6 cables on top of the Brocade? There is about a 1-2" clearance above the top-of-rack Brocade; and about 5-6 Cat6 cables would be resting on top the Brocade.

I don't think heat from the brocade would be a problem; but I like to ask before doing stuff. thanks for helping a server-rack noob out
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,668
113
31
fohdeesha.com
Odd question: re Brocade 6450-24p. I am just now filling my first-ever Server Rack Cabinet (12U, xRackPro2) which has front and rear doors (and side doors). The Brocade 6450 is top of rack, Rear.

Cable management question: is it 'safe' to feed Cat6 cables on top of the Brocade? There is about a 1-2" clearance above the top-of-rack Brocade; and about 5-6 Cat6 cables would be resting on top the Brocade.

I don't think heat from the brocade would be a problem; but I like to ask before doing stuff. thanks for helping a server-rack noob out
yeah there's nothing wrong with that

also thanks to @seatrope I realized the ICX6610 guide has been totally missing the "factory set-default" command, so anyone who had a used switch with a previous password was left to google when the guide told them to "enable" and it asked them for a PW they didn't have. The ICX6450 guide has had this instruction, not sure how I missed it on the 6610

added it - add missing password removal command · Fohdeesha/lab-docu@bc8d651
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,668
113
31
fohdeesha.com
loading up for the LAN party event this weekend: https://i.imgur.com/F4z8ecE.jpg

will try to post setup/event pics during this weekend. I think we may be the first large scale LAN party to offer 10gbE ports for attendees at every table, even 40gbE depending on which table you sit at :p

mostly ICX6610s, some ICX7250's, ICX6450s, and a couple FCX's for WAN agg to fiber in the telecom closet
 

tommybackeast

Active Member
Jun 10, 2018
290
111
43
yeah there's nothing wrong with that

also thanks to @seatrope I realized the ICX6610 guide has been totally missing the "factory set-default" command, so anyone who had a used switch with a previous password was left to google when the guide told them to "enable" and it asked them for a PW they didn't have. The ICX6450 guide has had this instruction, not sure how I missed it on the 6610

added it - add missing password removal command · Fohdeesha/lab-docu@bc8d651
thanks
 

tommybackeast

Active Member
Jun 10, 2018
290
111
43
loading up for the LAN party event this weekend: https://i.imgur.com/F4z8ecE.jpg

will try to post setup/event pics during this weekend. I think we may be the first large scale LAN party to offer 10gbE ports for attendees at every table, even 40gbE depending on which table you sit at :p

mostly ICX6610s, some ICX7250's, ICX6450s, and a couple FCX's for WAN agg to fiber in the telecom closet
Have a great LAN party, Guiness reports largest ever was an insane 22,180 people in Sweden, 2013

Largest LAN party

I see Guiness records for largest amount of people, highest elevation, longest in time; but nothing about "First Ever LAN Party with 10gbE and 40gbE" :)
 

seatrope

New Member
Oct 5, 2018
12
2
3
Maine
www.ychng.com
Thanks so much @fohdeesha for all your help. Greatly indebted. Icx 6610 humming along happily!

One dumb thing I did which would be obvious to most, but had me perplexed for a while. When I moved everything over from my old switch I ended up plugging in most things for a quick test, including several 10G DACs.

Soon after, everything on network started going crazy, unresponsive. PCs attached to 10G links started intermittently hanging. I realized that happened after I inserted the 10G DACs. On further investigation, of course I had looped a 10G cable and probably caused a broadcast storm...Duh.
 
  • Like
Reactions: fohdeesha