Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Sleyk · Mar 18, 2019

Thanks Argle, I hadn't thought about that

Will setup an alert now

arglebargle · Mar 18, 2019

Sleyk said:
Thanks Argle, I hadn't thought about that
Will setup an alert now

Sure thing. When I'm shopping for a bargain I usually do a quick skim of recent sales first, that tells you what the market price for the thing actually is. The confusing thing for a lot of people is that the prices you see for listings when you search on eBay aren't actual sale prices, they're the price at which things haven't sold yet. Definitely look at sold items first when you're considering a purchase, then set your alerts accordingly. Sometimes actual sale prices and current listings are waaaaaaay out of whack.

Blue)(Fusion · Mar 19, 2019

I think I got my 6610 ACLs figured out and removed almost all firewalling from pfSense to switch-duty. The only firwalling pfSense does now is to the WAN. All devices now use the switch VE interfaces as gateways. This is still new for me, so if anyone is curious to use it, be cautions and anyone with know-how, please tell me if this is wildly or even mildly wrong:

Code:

access-list 102 remark ALLOW DHCP
access-list 102 permit udp any any eq bootps                     
access-list 102 permit udp any any eq bootpc
access-list 102 remark ALLOW ANY ICMP
access-list 102 permit icmp any any
access-list 102 remark ALLOW ESTABLISHED TCP TRAFFIC
access-list 102 permit tcp any any established
access-list 102 remark ALLOW DNS REQUESTS TO PFSENSE
access-list 102 permit udp 10.1.2.0 0.0.0.255 host 10.1.2.254 eq dns
access-list 102 remark ALLOW NTP REQUESTS TO PFSENSE
access-list 102 permit udp 10.1.2.0 0.0.0.255 host 10.1.1.254 eq ntp
access-list 102 permit udp 10.1.2.0 0.0.0.255 host 10.1.2.254 eq ntp
access-list 102 remark DENY ALL OTHER ACCESS TO SWITCH AND ROUTER
access-list 102 deny ip any host 10.1.2.1 log
access-list 102 deny ip any host 10.1.2.254 log
access-list 102 remark DENY INTER-VLAN TRAFFIC
access-list 102 deny ip any 10.1.1.0 0.0.0.255
access-list 102 deny ip any 10.1.3.0 0.0.0.255
access-list 102 deny ip any 10.1.4.0 0.0.0.255
access-list 102 deny ip any 10.1.6.0 0.0.0.255
access-list 102 deny ip any 10.1.10.0 0.0.0.255
access-list 102 deny ip any 10.1.20.0 0.0.0.255
access-list 102 remark ALLOW SAME VLAN TRAFFIC
access-list 102 permit ip 10.1.2.0 0.0.0.255 10.1.2.0 0.0.0.255   
access-list 102 remark DENY REMAINING TRAFFIC
access-list 102 deny ip any any log
!
access-list 103 remark ALLOW DHCP
access-list 103 permit udp any any eq bootps
access-list 103 permit udp any any eq bootpc
access-list 103 remark ALLOW ANY ICMP
access-list 103 permit icmp any any
access-list 103 remark ALLOW ESTABLISHED TCP TRAFFIC
access-list 103 permit tcp any any established
access-list 103 remark ALLOW DNS REQUESTS TO PFSENSE
access-list 103 permit udp 10.1.3.0 0.0.0.255 host 10.1.3.254 eq dns
access-list 103 remark ALLOW NTP REQUESTS TO PFSENSE
access-list 103 permit udp 10.1.3.0 0.0.0.255 host 10.1.1.254 eq ntp
access-list 103 permit udp 10.1.3.0 0.0.0.255 host 10.1.3.254 eq ntp
access-list 103 remark ALLOW NTP REQUEST RETURNS FROM PFSENSE
access-list 103 permit udp host 10.1.1.254 eq ntp 10.1.3.0 0.0.0.255
access-list 103 permit udp host 10.1.3.254 eq ntp 10.1.3.0 0.0.0.255
access-list 103 remark ALLOW RETURN OF SNMP TRAFFIC TO LIBRENMS SERVER
access-list 103 permit udp 10.1.3.0 0.0.0.255 eq snmp host 10.1.6.61 gt 1024
access-list 103 permit udp 10.1.3.0 0.0.0.255 eq snmp-trap host 10.1.6.61 gt 1024
access-list 103 remark ALLOW ECOBEE STRICT WAN ACCESS
access-list 103 permit tcp host 10.1.3.50 host 216.220.61.236 eq 8190
access-list 103 remark DENY ALL OTHER ACCESS TO SWITCH AND ROUTER
access-list 103 deny ip any host 10.1.3.1 log
access-list 103 deny ip any host 10.1.3.254 log                   
access-list 103 remark DENY INTER-VLAN TRAFFIC
access-list 103 deny ip any 10.1.1.0 0.0.0.255
access-list 103 deny ip any 10.1.2.0 0.0.0.255
access-list 103 deny ip any 10.1.4.0 0.0.0.255
access-list 103 deny ip any 10.1.6.0 0.0.0.255
access-list 103 deny ip any 10.1.10.0 0.0.0.255
access-list 103 deny ip any 10.1.20.0 0.0.0.255
access-list 103 remark ALLOW SAME VLAN TRAFFIC
access-list 103 permit ip 10.1.3.0 0.0.0.255 10.1.3.0 0.0.0.255
access-list 103 remark DENY REMAINING TRAFFIC
access-list 103 deny ip any any log
!
access-list 104 remark ALLOW ICMP
access-list 104 permit icmp any any
access-list 104 remark DENY ALL OTHER ACCESS TO SWITCH
access-list 104 deny ip any host 10.1.4.1 log
access-list 104 remark ALLOW LOCAL VLAN TRAFFIC
access-list 104 permit ip 10.1.4.0 0.0.0.255 10.1.4.0 0.0.0.255
access-list 104 remark DENY REMAINING TRAFFIC
access-list 104 deny ip any any log
!
access-list 105 remark ALLOW DHCP
access-list 105 permit udp any any eq bootps
access-list 105 permit udp any any eq bootpc
access-list 105 remark ALLOW ANY ICMP
access-list 105 permit icmp any any
access-list 105 remark ALLOW ESTABLISHED TCP TRAFFIC
access-list 105 permit tcp any any established
access-list 105 remark ALLOW RETURN OF NTP REQUESTS FROM PFSENSE TO ANY VLAN
access-list 105 permit udp host 10.1.1.254 eq ntp 10.1.0.0 0.0.255.255
access-list 105 remark ALLOW RETURN OF SNMP TRAFFIC TO LIBRENMS SERVER
access-list 105 permit udp 10.1.1.0 0.0.0.255 eq snmp host 10.1.6.61 gt 1024
access-list 105 permit udp 10.1.1.0 0.0.0.255 eq snmp-trap host 10.1.6.61 gt 1024
access-list 105 remark ALLOW IPMI, DRAC ACCESS from VLAN10 and SLB1/2
access-list 105 permit udp 10.1.1.0 0.0.0.255 eq asf-rmcp 10.1.10.0 0.0.0.255 gt 1024
access-list 105 permit udp 10.1.1.0 0.0.0.255 eq asf-rmcp host 10.1.6.39 gt 1024
access-list 105 permit udp 10.1.1.0 0.0.0.255 eq asf-rmcp host 10.1.6.40 gt 1024
access-list 105 remark ALLOW RETURN TRAFFIC FROM PROXMOX HOSTS TO VLAN10
access-list 105 permit ip host 10.1.1.10 10.1.10.0 0.0.0.255
access-list 105 permit ip host 10.1.1.11 10.1.10.0 0.0.0.255
access-list 105 remark ALLOW IPERF3 TRAFFIC TO/FROM VLAN 10
access-list 105 permit tcp 10.1.1.0 0.0.0.255 10.1.10.0 0.0.0.255 eq 5201
access-list 105 remark DENY INTER-VLAN TRAFFIC
access-list 105 deny ip any 10.1.2.0 0.0.0.255
access-list 105 deny ip any 10.1.3.0 0.0.0.255
access-list 105 deny ip any 10.1.4.0 0.0.0.255
access-list 105 deny ip any 10.1.6.0 0.0.0.255
access-list 105 deny ip any 10.1.10.0 0.0.0.255
access-list 105 deny ip any 10.1.20.0 0.0.0.255
access-list 105 remark ALLOW REMAINING TRAFFIC
access-list 105 permit ip any any
!
access-list 106 remark ALLOW DHCP
access-list 106 permit udp any any eq bootps
access-list 106 permit udp any any eq bootpc
access-list 106 remark ALLOW ANY ICMP
access-list 106 permit icmp any any
access-list 106 remark ALLOW ESTABLISHED TCP TRAFFIC
access-list 106 permit tcp any any established
access-list 106 remark ALLOW DNS REQUESTS TO PFSENSE
access-list 106 permit udp 10.1.6.0 0.0.0.255 host 10.1.6.254 eq dns
access-list 106 remark ALLOW NTP REQUESTS TO PFSENSE
access-list 106 permit udp 10.1.6.0 0.0.0.255 host 10.1.1.254 eq ntp
access-list 106 permit udp 10.1.6.0 0.0.0.255 host 10.1.6.254 eq ntp
access-list 106 remark ALLOW LIBRENMS SERVER TO QUERY ALL VLANS
access-list 106 permit udp host 10.1.6.61 any eq snmp
access-list 106 permit udp host 10.1.6.61 any eq snmp-trap
access-list 106 remark ALLOW SLB1/2 SSH/222, IPMI, DRAC ACCESS TO ALL VLANS
access-list 106 permit tcp host 10.1.6.39 any eq ssh
access-list 106 permit tcp host 10.1.6.40 any eq ssh
access-list 106 permit tcp host 10.1.6.39 any eq rsh-spx
access-list 106 permit tcp host 10.1.6.40 any eq rsh-spx
access-list 106 permit tcp host 10.1.6.39 any eq asf-rmcp
access-list 106 permit tcp host 10.1.6.40 any eq asf-rmcp
access-list 106 permit udp host 10.1.6.39 any eq asf-rmcp
access-list 106 permit udp host 10.1.6.40 any eq asf-rmcp
access-list 106 remark ALLOW RETURN OF NFS UDP TRAFFIC FROM TORRENTS TO VLAN10
access-list 106 permit udp host 10.1.6.199 eq sunrpc 10.1.10.0 0.0.0.255
access-list 106 permit udp host 10.1.6.199 eq nfs 10.1.10.0 0.0.0.255
access-list 106 remark ALLOW IPERF3 TRAFFIC TO/FROM VLAN 10
access-list 106 permit tcp 10.1.6.0 0.0.0.255 10.1.10.0 0.0.0.255 eq 5201
access-list 106 remark DENY ALL OTHER ACCESS TO SWITCH AND ROUTER
access-list 106 deny ip any host 10.1.6.1 log
access-list 106 deny ip any host 10.1.6.254 log
access-list 106 remark DENY INTER-VLAN TRAFFIC
access-list 106 deny ip any 10.1.1.0 0.0.0.255
access-list 106 deny ip any 10.1.2.0 0.0.0.255
access-list 106 deny ip any 10.1.3.0 0.0.0.255
access-list 106 deny ip any 10.1.4.0 0.0.0.255
access-list 106 deny ip any 10.1.10.0 0.0.0.255
access-list 106 deny ip any 10.1.20.0 0.0.0.255
access-list 106 remark ALLOW REMAINING TRAFFIC
access-list 106 permit ip any any
!
access-list 110 remark VIP VLAN GETS ACCESS ANYWHERE
access-list 110 permit ip any any                                 
!
access-list 120 remark ALLOW DHCP
access-list 120 permit udp any any eq bootps
access-list 120 permit udp any any eq bootpc
access-list 120 remark ALLOW ANY ICMP
access-list 120 permit icmp any any
access-list 120 remark ALLOW ESTABLISHED TCP TRAFFIC
access-list 120 permit tcp any any established
access-list 120 remark ALLOW DNS REQUESTS TO PFSENSE
access-list 120 permit udp 10.1.20.0 0.0.0.255 host 10.1.20.254 eq dns
access-list 120 remark ALLOW NTP REQUESTS TO PFSENSE
access-list 120 permit udp 10.1.20.0 0.0.0.255 host 10.1.1.254 eq ntp
access-list 120 permit udp 10.1.20.0 0.0.0.255 host 10.1.20.254 eq ntp
access-list 120 remark ALLOW RETURN OF SNMP TRAFFIC TO LIBRENMS SERVER
access-list 120 permit udp 10.1.20.0 0.0.0.255 eq snmp host 10.1.6.61 gt 1024
access-list 120 permit udp 10.1.20.0 0.0.0.255 eq snmp-trap host 10.1.6.61 gt 1024
access-list 120 remark ALLOW HTTP/IPP/LPD/JETDIRECT TRAFFIC TO PRINTER
access-list 120 permit tcp 10.1.20.0 0.0.0.255 host 10.1.3.5 eq http
access-list 120 permit tcp 10.1.20.0 0.0.0.255 host 10.1.3.5 eq printer
access-list 120 permit tcp 10.1.20.0 0.0.0.255 host 10.1.3.5 eq ipp
access-list 120 permit tcp 10.1.20.0 0.0.0.255 host 10.1.3.5 eq 9100
access-list 120 remark DENY ALL OTHER ACCESS TO SWITCH AND ROUTER
access-list 120 deny ip any host 10.1.20.1 log
access-list 120 deny ip any host 10.1.20.254 log
access-list 120 remark DENY INTER-VLAN TRAFFIC
access-list 120 deny ip any 10.1.1.0 0.0.0.255
access-list 120 deny ip any 10.1.2.0 0.0.0.255
access-list 120 deny ip any 10.1.3.0 0.0.0.255
access-list 120 deny ip any 10.1.4.0 0.0.0.255
access-list 120 deny ip any 10.1.6.0 0.0.0.255
access-list 120 deny ip any 10.1.10.0 0.0.0.255
access-list 120 remark ALLOW REMAINING TRAFFIC
access-list 120 permit ip any any

Each ACL is numbered as 1${vlanid}

VLAN setup:
1 - Default - unused
2 - VoIP - no WAN access
3 - IoT - no WAN access - printers, cameras
4 - NAS/Gluster - no WAN access, no VLAN interface on pfSense)
5 - Management - access to all switch admin, pfSense, Proxmox hosts)
6 - App - VMs and other servers running various apps like MySQL, Apache, HAProxy, Deluged, monitoring software. Some with external access from WAN (HTTP/HTTPS)
10 - Trusted - My desktops, HTPC, laptop
20 - Untrusted - Guest wireless devices, cell phones, iPads, Smart TVs

Also, all the access-groups are in the "in" direction. Is this appropriate?

Code:

interface ve 2
 ip access-group 102 in
 ip address 10.1.2.1 255.255.255.0
!
interface ve 3
 ip access-group 103 in
 ip address 10.1.3.1 255.255.255.0
!
interface ve 4
 ip access-group 104 in
 ip address 10.1.4.1 255.255.255.0                              
!
interface ve 5
 ip access-group 105 in
 ip address 10.1.1.1 255.255.255.0
!
interface ve 6
 ip access-group 106 in
 ip address 10.1.6.1 255.255.255.0
!
interface ve 10
 ip address 10.1.10.1 255.255.255.0
!
interface ve 20
 ip access-group 120 in
 ip address 10.1.20.1 255.255.255.0

ekv · Mar 19, 2019

I'm not a networking guy. Two months ago (on a whim) I picked up a MikroTik CRS326 and ConnectX-2 cards, and the SFP+ just works. Shocked, I immediately started looking for more and affordable SFP+ ports. Hence looked at LB6M though bought Brocade icx6450-12p (hoping for lower power consumption).

I go to "Fohdeesha Docs" -- which is mind-bogglingly, take-my-money-please amazing -- and one of the first things is "Connect to the switches serial/console port..."

Umm, ok.

This may be a (very) dumb question, but which is the serial port? I've looked through the 6450 docs and ... there appears to be no labeling, just icons. I'm guessing the top (left-most) port is the serial, and the one just below it is the mgt. port.

The next question then is, since my computer(s) has no serial port, how do I connect?

There's this
USB to RS232 Serial to RJ45 CAT5 Console Adapter Cable for Cisco Routers FTDI | eBay
But is that actually converting anything? [I literally have no COM ports in my (w10) Device Mgr.] So would I instead need a combo RJ45-to-DB9 cable and then a DB9-to-USB cable? Like this

TRENDnet USB to Serial Converter, USB 1.1 to RS-232 Male (9-pin) DB9 Serial | eBay

9-Pin DB9 Serial RS232 Port to RJ45 Cat5 Ethernet LAN Rollover Console 797734232417 | eBay

Thanks in advance. [Already installed PuTTy and Tftp via Chocolatey]

KFJ · Mar 19, 2019

I recently purchased a used ICX6610-48P-PE with the PREM and 10G licences. If I update the firmware to latest, will the licences still be valid/unaffected? Do I need to back up the licences prior to updating the firmware?

arglebargle · Mar 19, 2019

arglebargle said:

Does anyone know how to trap chassis notifications? The Fan Up/Down notifications are making it hard to concentrate on configuring my switches.

This is what I'm talking about:

Code:

Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up

core02(config)#
core02(config)#Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up

fohdeesha said:
what switch model is that? I've never seen fan up/down messages on any brocade model unless I physically pulled out a fan tray during operation, does one of your fan trays have a loose connection or something?

So I investigated this a bit more today and as far as I can tell there's no issue with the fan actually having a loose connection, the firmware just perceives that the fan is down. I ran the switch with the cover off on my desk for an hour and the fan was definitely spinning while the firmware declared that it was failed.

The only thing I can think of here is that my silent fan is running at too low an RPM for the switch to perceive it as working. I guess I can test with the slightly higher RPM fan that I bought to use with my -24P but never ended up installing.

Is there a way to stop the switch from sending logs to the console connection but keep the notices in the log? That'd at least let me work on the CLI without these interruptions. For the moment I'm just using "no logging buffered debug" and "no logging buffered informational" ... it just seems like a dumb idea to add "no logging buffered alerts" in case something actually breaks.

fohdeesha · Mar 20, 2019

arglebargle said:
So I investigated this a bit more today and as far as I can tell there's no issue with the fan actually having a loose connection, the firmware just perceives that the fan is down. I ran the switch with the cover off on my desk for an hour and the fan was definitely spinning while the firmware declared that it was failed.

The only thing I can think of here is that my silent fan is running at too low an RPM for the switch to perceive it as working. I guess I can test with the slightly higher RPM fan that I bought to use with my -24P but never ended up installing.

Is there a way to stop the switch from sending logs to the console connection but keep the notices in the log? That'd at least let me work on the CLI without these interruptions. For the moment I'm just using "no logging buffered debug" and "no logging buffered informational" ... it just seems like a dumb idea to add "no logging buffered alerts" in case something actually breaks.

are they getting sent to the physical serial port or the telnet/ssh session? Either way probably no way to silence them, on purpose, I can't think why you would ever want to not want to know about a critical piece of the switch failing (other than well, because you replaced the fans and it's wrong

)

you could just cut the RPM sense wire, that way it'll think it has failed right off the bat on boot and never report again. Or get a faster fan

fohdeesha · Mar 20, 2019

ICX6650

$300 (or even less occasionally)
64x 10gbE (56 via front SFP+, 8 via 2x rear QSFP breakout)
4x 40gbE via rear QSFP
120w power draw
a little louder than the icx6610
all the advanced software features of the icx6610 (vrf's, bgp, tunnels, etc)
takes same PSU and fan tray models as the ICX6610 (same rack ears too)
does NOT support stacking
DOES support MC-LAG (LACP across switches) (Brocades term is MCT)

a little much for most home users, but a good deal if you have a lot of 10gb to aggregate

this seller accepted a BO of $250. It was listed and shown as no rack ears, but came with some. Can't promise his remaining stock will as well - Brocade ICX6650 56-Port 10/40 Gigabit Ethernet Top-of-Rack Switch 42111395699 | eBay

soul extraction:

Matt G · Mar 20, 2019

I'm not sure what I'm getting myself into but this thread was just so great I've ordered three icx6610 from two different sellers... all the way from the states to the north ass-end of Sweden. I guess I should start reading the "fine print" on page one before I get burnt to a crisp for stupid questions. From what I understand the last two I ordered should have atleast one 4-port license upgrade for the sfp ports.

I sort of got into to this because I got tired of my synology-devices being crap. I bought two Dell r720xd, followed by some reading just to realize I can't keep doing 1gbe anymore because... upgrades! So now I'm stuck with more ambitions than time. I haven't kept up with progress since the late '90s, I guess I'm jumping in the deep end here.

Now I'm off to rtfm...

And big thanks to @fohdeesha for this most excellent thread/post/whatever... I'm swedish, no habla ingles...

Zervun · Mar 20, 2019

Matt G said:
I'm not sure what I'm getting myself into but this thread was just so great I've ordered three icx6610 from two different sellers... all the way from the states to the north ass-end of Sweden. I guess I should start reading the "fine print" on page one before I get burnt to a crisp for stupid questions. From what I understand the last two I ordered should have atleast one 4-port license upgrade for the sfp ports.

I sort of got into to this because I got tired of my synology-devices being crap. I bought two Dell r720xd, followed by some reading just to realize I can't keep doing 1gbe anymore because... upgrades! So now I'm stuck with more ambitions than time. I haven't kept up with progress since the late '90s, I guess I'm jumping in the deep end here.

Now I'm off to rtfm...

And big thanks to @fohdeesha for this most excellent thread/post/whatever... I'm swedish, no habla ingles...

Page one around post 6 has info for the licenses. You shouldn't need licenses when ordering them. Need to email fodesha

tjk · Mar 20, 2019

fohdeesha said:
ICX6650

$300 (or even less occasionally)

56x 10gbE (48 via front SFP+, 8 via 2x rear QSFP breakout)

4x 40gbE via rear QSFP

115w power draw

a little louder than the icx6610

all the advanced software features of the icx6610 (vrf's, bgp, tunnels, etc)

takes same PSU and fan tray models as the ICX6610 (same rack ears too)

does NOT support stacking

DOES support MC-LAG (LACP across switches) (Brocades term is MCT)

a little much for most home users, but a good deal if you have a lot of 10gb to aggregate

this seller accepted a BO of $250. It was listed and shown as no rack ears, but came with some. Can't promise his remaining stock will as well - Brocade ICX6650 56-Port 10/40 Gigabit Ethernet Top-of-Rack Switch 42111395699 | eBay

soul extraction:

My secret is out!

Did you get the license "stuff" working with these yet?

fohdeesha · Mar 20, 2019

tjk said:
My secret is out!

Did you get the license "stuff" working with these yet?

yep, hence the post. Wanted to include them in the original post nearly a year ago but they're nearly useless without licenses :/

Emdee · Mar 20, 2019

I'm looking into actually hooking up my Dell R720 to my 6610 and after searching through the many pages in this thread I came to the following conclusion: Everything is a lot more expensive in Europe.

Would buying these two be a smart choice?
Mellanox ConnectX-3 CX354A PCIe x8 40 GBe QSFP+ Dual Port Server HP 649281-B21 | eBay
Brocade 40G-QSFP-4SFP-C-0301 40G QSFP+ Breakout DAC Cable

I can't really find a Mellanox ConnectX-3 cheaper in the Netherlands / EU, same for the cable so I figured I'd get a new one.

Would this be a good combo, or is there a better way to do this?

fohdeesha · Mar 20, 2019

the card looks good, but that cable is an SFP+ breakout cable, when it sounds like you want to do 40gb direct to that mellanox card in your dell, in which case you need qsfp to qsfp like this NEW Brocade 58-0000033-01 4x10GE QSFP Passive Cable 1M | eBay

arglebargle · Mar 20, 2019

fohdeesha said:
are they getting sent to the physical serial port or the telnet/ssh session? Either way probably no way to silence them, on purpose, I can't think why you would ever want to not want to know about a critical piece of the switch failing (other than well, because you replaced the fans and it's wrong )

you could just cut the RPM sense wire, that way it'll think it has failed right off the bat on boot and never report again. Or get a faster fan

They're going to the physical serial port, I have a pi plugged into the same switch that the 6450 is uplinked to so I can faf about with the thing and not lock myself out.

Here's some fan data for anyone shopping for quiet 6450 replacements

https://media.digikey.com/pdf/Data Sheets/Sunon PDFs/MagLev_Motor_Fan.pdf

The KDE1204PKV3 (0.4W) generates intermittent "Fan Up, Fan Down" logs but works fine otherwise.
The MB40201VX (KDE1204PKVX redesign) (1.4W) works fine.

It's tricky sourcing the V1 without paying the full Digikey price but I've got a couple of V2 (0.8W) fans on the way to test too.

i386 · Mar 20, 2019

Emdee said:
I'm looking into actually hooking up my Dell R720 to my 6610 and after searching through the many pages in this thread I came to the following conclusion: Everything is a lot more expensive in Europe.

Would buying these two be a smart choice?
Mellanox ConnectX-3 CX354A PCIe x8 40 GBe QSFP+ Dual Port Server HP 649281-B21 | eBay
Brocade 40G-QSFP-4SFP-C-0301 40G QSFP+ Breakout DAC Cable

I can't really find a Mellanox ConnectX-3 cheaper in the Netherlands / EU, same for the cable so I figured I'd get a new one.

Would this be a good combo, or is there a better way to do this?

Are you looking for connect-x3 cards just in the netherlands?
If you don't mind to wait you can get them cheaper from the states: HP 544QSFP Infiniband 10/40GB FDR/EN PCIe Dual Port HCA 656089-001 649281-B21 | eBay

arglebargle · Mar 20, 2019

I watched a pair of Sun branded C-X3's go for around $30 this morning as well. Let me find a link to their OEM part number.

This is ended but it has the part numbers you're looking for:
LOT 2 Sun Oracle 7046442 40GB+10Gb Dual QSFP Infiniband Connect PCI-Express Card | eBay

fohdeesha · Mar 20, 2019

KFJ said:
I recently purchased a used ICX6610-48P-PE with the PREM and 10G licences. If I update the firmware to latest, will the licences still be valid/unaffected? Do I need to back up the licences prior to updating the firmware?

firmware updates do not affect licenses. and if in some freak accident you lose them, pm me and I'll send more

fohdeesha · Mar 20, 2019

ekv said:
I'm not a networking guy. Two months ago (on a whim) I picked up a MikroTik CRS326 and ConnectX-2 cards, and the SFP+ just works. Shocked, I immediately started looking for more and affordable SFP+ ports. Hence looked at LB6M though bought Brocade icx6450-12p (hoping for lower power consumption).

I go to "Fohdeesha Docs" -- which is mind-bogglingly, take-my-money-please amazing -- and one of the first things is "Connect to the switches serial/console port..."

Umm, ok.

This may be a (very) dumb question, but which is the serial port? I've looked through the 6450 docs and ... there appears to be no labeling, just icons. I'm guessing the top (left-most) port is the serial, and the one just below it is the mgt. port.

The next question then is, since my computer(s) has no serial port, how do I connect?

There's this
USB to RS232 Serial to RJ45 CAT5 Console Adapter Cable for Cisco Routers FTDI | eBay
But is that actually converting anything? [I literally have no COM ports in my (w10) Device Mgr.] So would I instead need a combo RJ45-to-DB9 cable and then a DB9-to-USB cable? Like this

TRENDnet USB to Serial Converter, USB 1.1 to RS-232 Male (9-pin) DB9 Serial | eBay

9-Pin DB9 Serial RS232 Port to RJ45 Cat5 Ethernet LAN Rollover Console 797734232417 | eBay

Thanks in advance. [Already installed PuTTy and Tftp via Chocolatey]

if you bought the ICX 6450-C12, the smaller white thing, be aware it has no 10gb ports - it's not listed in the original post for a reason. The first cable you linked will indeed work, it's a usb to serial converter with the proper rj45 end that our switches have. The serial port is the one labeled with ones and zeroes like "IOIO", the ethernet management port is the one with the network/computer icon

ekv · Mar 20, 2019

fohdeesha said:
if you bought the ICX 6450-C12

D'oh! brain fade, sorry. ICX 6450-24p.

fohdeesha said:
The first cable you linked will indeed work.... The serial port is the one labeled with ones and zeroes like "IOIO", the ethernet management port is the one with the network/computer icon

Bought 'em all yesterday. Figured something would eventually stick to the wall (as it were 8)

serial on top, mgt right below. Got it. Many thanks.

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Your Friendly Knowledgable Helper and Techlover!

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

Active Member

New Member

New Member

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

Kaini Industries

Kaini Industries

New Member

Member

Active Member

Kaini Industries

New Member

Kaini Industries

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

Well-Known Member

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

Kaini Industries

Kaini Industries

New Member