Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

DMFDMinister

New Member
Oct 4, 2020
4
10
3
absolutely - what's riveted to the chassis is the nut those things are screwing into - they're the same as motherboard standoffs, just longer. the solder around all the circles is so the PCB makes good electrical contact with the standoffs (and therefore chassis ground), it's not actually bonded to the standoff
Well that's helpful - and leaves me feeling a bit sheepish. Now just a matter of figuring out how I'm going to mod this. I'm curious if better compound with slightly stronger push-pin mount with 9-10K RPM fans will keep the fans operating in speed 1 while still satisfying the tach requirements. Still going to do some digging on the heatsink and if there's something better I can install.
 
  • Love
Reactions: virulent

rootwyrm

Member
Mar 25, 2017
74
93
18
www.rootwyrm.com
Annoyed slightly by this issue, I started thinking about re-applying thermal paste - because I didn't quite understand how my switch could idle at 58C on the ASIC with 68-70F ambient temps when I literally only had a serial cable plugged into it and nothing else. I think the thing deterring people from doing this is that it's very difficult to do without damaging the switch. The heat sink is mounted with push pins that typically require you to have access to the other side of the board to get them out. Once the pins are in, they're not coming out without some serious persuasion.
Because it's designed to operate at those temperatures. 24x7x365. For years. These are not consumer junk. The max die on these processors is 105C+, all of the capacitors are 85C rated or above, and it is designed to run at these temperatures and higher without failing. They get thrown into the top of a rack with terrible airflow design at best, with 30U+ of heat source directly below, and next to no cold air supply. And that's the least severe duty they do.
And believe me, I would know. I've been engineering the systems that rack below these for decades. Brocade's one of the few I'm not aware of having successfully cooked out when located in the same rack as one of my heat exchanger equipped systems. Even when basically starved of cold air supply and sitting in a rack with more than 8kW draw, it was fine.

Anyone know where I can get specs on this heatsink? I imagine re-applying good paste to the heatsink (like MX4 thermal compound or ceramique) could go a long ways here to improving temperatures given the previous situations.
That's not thermal paste. That's a perfectly good reusable thermal pad, which you ruined. They get used in this sort of application because they do not wear out or dry out. (No, that was not dried out. It was perfectly fine till you separated it from the woven layer.) And they're more tolerant of severely unlevel IHS and CPU mounting. That IHS gets slapped on there with some glue to protect the die and give them somewhere for a pretty part label, not be a critical heat transfer element.
And no, you can't just buy replacement. The particular compound they used there is familiar to me, and only sold in bulk roll. And no, you can't just slap some AS5 or GC12 on and 'fix' it; the woven compound is impregnated.
Thermal compound is not going to improve performance meaningfully at all. These are very, very low TDP parts. All the heat load is in the POE board. You could strap an Alpha cast copper slug on there and it wouldn't net you anything significant. Which is why they're just fine with a cheap, loose coupled, cast aluminum part that isn't even in direct airflow but go to pains to route over the POE board. So basically anything that matches the pin span (which is measured center to center with a caliper) and fits within the keepout zone indicated by the outer white box will work fine.
No, really. Whatever. Doesn't matter. It'll be fine. Effectively coupling will be a whole other story, but, it won't break anything.
 

pokeimon

New Member
Oct 13, 2019
10
1
3
USA
I'm a relatively new to this and I have run up against a wall of what to do next.
I'm trying to configure the switch as layer 3 while routing outgoing traffic to OPNsense (More or less PFsense).

Port 1/2/1 which is connected to OPNsense with a Gateway created with the IP set to 10.0.2.1 and a Route set to that gateway with Network set to 10.0.10.0/24 to encompass the VLAN 10 network. OPNsense's IP is 10.0.2.2.

I did try to follow a guide though its for a CISCO router but I'm stuck.
I am currently just trying to get VLAN 10 to connect to OPNsense/Internet. Currently I created VLAN 2 to be used as the Transit VLAN and VLAN 10 to be a 'trusted' network.

Any help would be much appreciated.

For the switch I have the following config:
Code:
ICX6450-48P Router(config)#show run
Current configuration:
!
ver 08.0.30tT313
!
stack unit 1
  module 1 icx6450-48p-poe-port-management-module
  module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 2 name Transit by port
 untagged ethe 1/1/1 ethe 1/2/1
 router-interface ve 2
!
vlan 10 name Trusted by port
 untagged ethe 1/1/3
 router-interface ve 10
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
ip dhcp-client disable
ip dhcp-server enable
!
ip dhcp-server pool pool10
 dhcp-default-router 10.0.10.1
 dns-server 1.1.1.1
 excluded-address 10.0.10.1
 excluded-address 10.0.10.2 10.0.10.99
 lease 1 0 0
 network 10.0.10.0 255.255.255.0
 deploy
!
ip route 0.0.0.0/0 10.0.2.2
!
no telnet server
username root password .....
!
!
!
!
!
interface ve 2
 ip address 10.0.2.1 255.255.255.0
!
interface ve 10
 ip address 10.0.10.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end
 

BobTB

Member
Jul 19, 2019
81
19
8
did you run the "ip mtu 9000" command on the ve assigned to that vlan? should work fine. if you're trying to do it just on a per port basis I believe you'll need to run the layer2 only firmware
Ok, I think I found out what is wrong. This is bad. Very bad. I am using mikrotik sfp+ ports into rj45 10gbE copper adapters ( S+RJ10 )

These do not support jumbo frames? Check the forum posts below:


This is the only thing that could be wrong with my setup, I tried everything to make it work, and it does not :( work

Anyone else using these sucesfully with jumbo frames?
 
Last edited:

BobTB

Member
Jul 19, 2019
81
19
8
ok, there are two versions rev.1 and rev.2 of Mikrotik S+RJ10. The R2 supports jumbo, R1 does not. I don't even need to go to look on mine which rev. they are....
 
  • Like
Reactions: fohdeesha

Dreece

Active Member
Jan 22, 2019
503
160
43
Just thought I'd do a catch up on this thread, wow have things moved on! Some great mods going on!!

I really think this thread needs a sub forum, "Brocade/Ruckus Enthusiasts" or something.

224 pages and counting o_O
 
  • Like
Reactions: fohdeesha

virulent

Member
Jul 3, 2019
28
39
13
I'm a relatively new to this and I have run up against a wall of what to do next.
I'm trying to configure the switch as layer 3 while routing outgoing traffic to OPNsense (More or less PFsense).

Port 1/2/1 which is connected to OPNsense with a Gateway created with the IP set to 10.0.2.1 and a Route set to that gateway with Network set to 10.0.10.0/24 to encompass the VLAN 10 network. OPNsense's IP is 10.0.2.2.

I did try to follow a guide though its for a CISCO router but I'm stuck.
I am currently just trying to get VLAN 10 to connect to OPNsense/Internet. Currently I created VLAN 2 to be used as the Transit VLAN and VLAN 10 to be a 'trusted' network.

Any help would be much appreciated.

For the switch I have the following config:
Code:
ICX6450-48P Router(config)#show run
Current configuration:
!
ver 08.0.30tT313
!
stack unit 1
  module 1 icx6450-48p-poe-port-management-module
  module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 2 name Transit by port
untagged ethe 1/1/1 ethe 1/2/1
router-interface ve 2
!
vlan 10 name Trusted by port
untagged ethe 1/1/3
router-interface ve 10
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
ip dhcp-client disable
ip dhcp-server enable
!
ip dhcp-server pool pool10
dhcp-default-router 10.0.10.1
dns-server 1.1.1.1
excluded-address 10.0.10.1
excluded-address 10.0.10.2 10.0.10.99
lease 1 0 0
network 10.0.10.0 255.255.255.0
deploy
!
ip route 0.0.0.0/0 10.0.2.2
!
no telnet server
username root password .....
!
!
!
!
!
interface ve 2
ip address 10.0.2.1 255.255.255.0
!
interface ve 10
ip address 10.0.10.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end
your switch config looks ok (I think?)
from a device on vlan10 (not opnsense) can you ping the switch and opnsense? 10.0.10.1 (will definitely work), 10.0.2.1 (should), 10.0.2.2 (should)

if you can ping 10.0.2.2 then you have an issue on OPNsense, so check your NAT rules that they are OK. Also check the live firewall view for any blocked packets from 10.0.10.0/24.

edit: tied to above, does it work with a device on vlan2?
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
ip helper-address: this is your DHCP server. The IP is the native host IP in its native VLAN! This is essentially the DHCP relay.
Good writing, but just to clarify... the switch itself in this case is the DHCP Relay, the helper address points to the server that receives the packet sourcing from the VE IP address on behalf of the client asking for an address. ;) The server can choose the right subnet/pool based of the source address (ie. the VE one).
 
  • Like
Reactions: virulent

pokeimon

New Member
Oct 13, 2019
10
1
3
USA
your switch config looks ok (I think?)
from a device on vlan10 (not opnsense) can you ping the switch and opnsense? 10.0.10.1 (will definitely work), 10.0.2.1 (should), 10.0.2.2 (should)

if you can ping 10.0.2.2 then you have an issue on OPNsense, so check your NAT rules that they are OK. Also check the live firewall view for any blocked packets from 10.0.10.0/24.

edit: tied to above, does it work with a device on vlan2?
Thanks for the info!
It was my NAT that was incorrectly set up (or I should say forgotten to set up)
 
  • Like
Reactions: virulent

koifish59

Member
Sep 30, 2020
66
19
8
I just received a ICX 6610!

I don't have a console cable yet. While I wait for one, if I do a factory reset, will the switch be ready to use on my network as a plug-and-play unmanaged switch?
 

virulent

Member
Jul 3, 2019
28
39
13
I just received a ICX 6610!

I don't have a console cable yet. While I wait for one, if I do a factory reset, will the switch be ready to use on my network as a plug-and-play unmanaged switch?
More or less, except your switch won't have a management IP (or does it default to DHCP on the mgmt port with telnet enabled? ...no idea). It will only route VLAN 1 but will otherwise act as unmanaged. PoE (if equipped) is off by default so if needed you will need to wait for a console cable.
 
  • Like
Reactions: koifish59

klui

Well-Known Member
Feb 3, 2019
824
453
63
how do you plan on factory resetting it without a console cable?
This never even entered my mind and I would have thought the same as @koifish59. Recessed reset buttons can almost always be used to perform a factory default. But the ICX6610 installation guide says otherwise:

Reset button
The reset button allows you to restart the system without switching the power supplies off and on or using the CLI or Web Management Interface. When the reset button is pressed, the system resets and the software is reloaded. The reset button is located next to the PSU LED on both 24-port and 48-port models.

The de facto behavior was only added to 8.0.80 and the 6610 can't run anything newer than 8.0.30.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,727
3,075
113
33
fohdeesha.com
Indeed, if there was a physical way of wiping the config I would have been able to make the guides much simpler (no console cable needed, reset to default and plug management port in, telnet to the IP it grabs via dhcp)
 

koifish59

Member
Sep 30, 2020
66
19
8
how do you plan on factory resetting it without a console cable?
Oh I guess I assumed too much! I saw youtube vid saying you can physically reset a brocade switch by pressing and holding down the physical reset button, plug in the power cable, wait 10 seconds, then it should wipe everything to factory default.

One of the engineers for brocade. So this only applies to the ICX 7000 series?