Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tfran1990

New Member
Jul 19, 2019
3
1
1
I used the guide in the very beginning of this thread and i was able to configure everything as far a ve and vlans with the attached ports. my problem is i cant ping a laptop connected to one of the vlans. is this the right place to post a config?
 

kpfleming

Active Member
Dec 28, 2021
391
205
43
Pelham NY USA
In short, people are stupid and shortsighted. Most of the problems in that blog rant are either unsolvable, irrelevant or "thinking is hard". I have had working IPv6 for nearly a decade - using a HE.net tunnel, because most ISPs won't provision IPv6 in Australia. And I'm by no means a network engineer.
Same here, I've had IPv6 in my home network (in multiple variations) for years and it works well and solves actual problems.

Back to the OP's question: with the configuration you have, you'll need to setup IPv6 address configuration and routing at each hop, you can't expect RAs to *pass through* layer 3 network elements (routers). RAs are *local* configuration, they don't pass through routers.

Assuming that FIOS allows you to use DHCPv6-PD to get a prefix delegation of reasonable size (/56 or larger), which they don't yet in my FIOS neighborhood, you'll need to do various things:

* The OPNsense firewall will need to obtain a delegation from FIOS using DHCPv6-PD. It may also get its own address (via SLAAC or DHCPv6) but that is not mandatory, as it will already have a link-local address that can be used for IPv6 transit between itself and the FIOS network. The delegation will need to be at least a /56 in size, although a /48 would be better given your configuration; if it's a /64, you're stuck because that can't be broken down further.

* The 7250 'core router' will need to be given a block of addresses it can use on the VLANs it manages. Just like the FIOS-OPNsense link there is no need for addresses to be assigned to the transit LAN, that LAN can use the automatic link-local addresses. The block can be given to the 7250 via DHCPv6-PD *from OPNsense* (not FIOS) if OPNsense is able to do that; if not, it can be manually configured on both ends.

* Each of the VLANs on the 7250 will need to be manually configured with a sub-block out of the 7250's block (but no smaller than /64). The 7250 would then be configured to emit RAs on the LANs so that hosts can get IPv6 addresses and have routing information. It can also provide DHCPv6 services if you desire, but that is not necessary.

* There is no need for any dynamic routing protocols for this to work; the subnet addresses form a 'tree' starting from the block delegated by FIOS, and each router in the tree knows about the downstream routers, but there's no need for them to know about 2nd or 3rd level routers/routes below them.

* "passing through" DHCPv6 traffic via the OPNsense firewall isn't likely to work out well. That sort of thing can work OK for host addresses (not prefix delegation) if the device doing the passthrough supports NDP proxying (similar to ARP proxying in IPv4), but that isn't going to work with DHCPv6-PD.
 

Oodaloop

New Member
Jul 28, 2021
3
0
1
I am having connectivity problems hooking up my ICX-7250 to a Dell Poweredge R740 server . The Dell has an Intel X520/I350 daughter card. My connection uses a DAC cable from one of the 10G ports on the Intel card to one 10G port on the ICX-7250.
My connection is very instable , with frequent loss of connectivity. Not sure how to troubleshoot this - change the DAC cable? I previously had stp setup on the brocade, so I turned it off because I thought the Dell Poweredge was bringing down the network with a broadcast storm. The network was rock solid until I included the Dell server. What should I be looking for?
Did you ever get this figured out? I'm having a similar issue with a R710 server. The 10G ports on the ICX work when initially booted then just shutdown eventually and I can't get them to turn back on until after a reboot.

Anyone else have this experience or could walk me through some troubleshooting steps to figure out the problem or atleast a way to get the ports back on without having to power cycle the entire switch?
 

Vesalius

Active Member
Nov 25, 2019
253
195
43

LodeRunner

Active Member
Apr 27, 2019
540
227
43
I'm trying to hunt down packet loss in a stack of 7450's. Work has a stack of 6, full ring using QSFP modules in the back, and up until a few days ago everything was fine.

Now, despite a full stack reload and reflash to 8095g we're getting measurable packet loss to many hosts across all the units in the stack. Ruckus has a document about using the MAC filter to identify packet loss, but it's locked behind paid support. Any advice?

Edit: sh int and sh stats on the stack ports show all zeroes in the error counters. Ports used for testing area also showing no errors.

Edit 2: symptoms may have actually begun a week ago and presented as a reporting service generating blank reports (reporting service finalizes the file after receiving data or timing out). In addition to a full stack reload yesterday and a ISSU upgrade today, we reloaded all downstream switches in the building.


We traced it all to a workstation; once that port was disabled, we stopped losing traffic. Plugging that workstation into a different port immediately created the issue again. So diagnosing the home run and the workstation NIC will be fun.
 
Last edited:
  • Like
Reactions: Lone Wolf

heromode

Active Member
May 25, 2020
380
201
43
I'm not happy with the chinese fans on the ASIC's, they're actually creating noise that can be heard outside the case. Onto the next option, 3x Noctua nf-a4x10-flx fans with low noise adapter on the asic's, spinning at 3700 RPM, and fed from the 12V rail of the PSU. Then maybe 1x Noctua NF-A4x20 FLX at the rear also fed with 12V and a low noise adapter, to slightly increase the outbound airflow, because the Arctic S4028-6K's barely move any air at 4.5V.

I'm bummed that i wasn't able to get the 24 port ICX6450, thermal management would have been so much easier with 25W. I ordered the last one but had to pay with bank transfer, and while the seller was waiting for that transfer, some
yahoo came along and bought the last 24 version with paypal, and the seller then offered me the 48 at the same price.

Now looking at the markets i see the 48 port models are plentiful and selling for cheap, but the 24 port models are few and far between, and much more expensive. Man i wish i could lose that extra 25W idle :(

ATM at idle and the summer heatwave gone, i'm running at 55 degrees in a small room with lots of old gear producing heat, which will all soon be gone. But i'm determined to make my 48p model absolutely silent, it's just the 25 Watts running 24/7 is gonna cost me about 25 EUR annually until december 2024, and after that, who knows..

edit anyone have a spare good condition 6450-24, poe or no poe, and wanna swap with a -48p in excellent condition in europe, lemme know.
 
Last edited:

heromode

Active Member
May 25, 2020
380
201
43
Does anyone know if a 2.5Ggbe SFP+ to RJ45 adapter will work in the ICX6450?
@zanechua It seems that the modules with the real Aquantia chipset have special capabilities for allowing the intermediate speeds of NBase-T on switches like the Brocade ICX that do not clock at those intermediate speeds. See https://forums.servethehome.com/ind...t-marvell-88x3300-v-s-aquantia-aqs-107.30004/ for a more thorough explanation. Also https://forums.servethehome.com/index.php?threads/mix-10g-2-5g-slow-speed-high-retr.33217/.

Using your HiFiber tranceiver, do you get full 2.xGbps throughput in BOTH directions between a 10Gbe endpoint and a 2.5Gbe endpoint?
@PANiCnz i don't know the answer to your question, but the post above should give you some idea..

edit i remember from the thread that any standard 2.5Gbe will NOT work, it needs to be some special transceiver that presents itself as a 10gbit to the switch, and does the speed conversion internally.

The ICX6450 itself does NOT support 2.5gbe.
 
Last edited:

tdk

New Member
Aug 31, 2022
2
1
3
Hi,

Just made it through 379 pages... skimming most of it...
Thank you @fohdeesha for your generosity.

As a result I got some 6610, 6450s. I am looking forward to playing with them.

Has anyone mounted the 6610 sideways? Because of space constraint, the rack and patch panel will not accommodate the 6610-48 (I can do 6450-24 x 2.. but would prefer 6610-48 x 1). I can mount it sideways in something like this:


Yes, no?

Thank you.
 

donedeal19

Member
Jul 10, 2013
38
12
8
@PANiCnz I have an 7250-48p, but I have an qsfptek sfp+ for ubiquiti. Usb type c to 2.5gbe from pc to sfp+ shows up connected.
Also shows up on the 7150-c12p as a quick test. Not sure on how the performances just plug and play.
 

bluez

New Member
Sep 4, 2022
2
0
1
I was eyeing the ICX6610-48P-E for home setup. I have a few 10GbE capable devices and this port has PoE+ so it fits the bill well. However, I need help to understand how to use the QSFP+ ports to add 10GbE RJ45 copper? I am a newbie and a bit lost here.
 

SlothCroissant

New Member
Sep 4, 2022
1
0
1
Hey folks, long time lurker, first time poster. I've scanned through this thread and have gotten mixed results (some people saying their 6610s ramp the fans to a million RPM all the time, others who never see north of 40C), and I think I just need to ask flat out:

Got an ICX6610 that I've been running for a few months now, and most of my learning has been from this thread (came from a Cisco C3750X-48P to a ICX6610-48P). Quick question - what do people see as far as temps are concerned with these? Mine seems to sit at 75C in a room that is around 72F-74F, and I can't figure out why it's so hot. I've heard it runs a bit spicy, but I didn't think it lived at its fan speed switching point.

Code:
SSH@ff-icr01.lan>sh chassis
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
     Model Number:    23-0000142-02
    Serial Number:    DPZ
    Firmware Ver:      B
Power supply 1 Fan Air Flow Direction:  Front to Back
Power supply 2 (AC - PoE) present, status ok
     Model Number:    23-0000142-02
    Serial Number:    F3N
    Firmware Ver:      B
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 72.0 deg-C

Fan speed switching temperature thresholds:
        Speed 1: NM<----->76       deg-C
        Speed 2:       71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back
MAC 1 Temperature Readings:
    Current temperature : 53.5 deg-C
MAC 2 Temperature Readings:
    Current temperature : 63.5 deg-C
CPU Temperature Readings:
    Current temperature : 72.0 deg-C
sensor A Temperature Readings:
    Current temperature : 53.0 deg-C
sensor B Temperature Readings:
    Current temperature : 58.5 deg-C
sensor C Temperature Readings:
    Current temperature : 43.5 deg-C
stacking card Temperature Readings:
    Current temperature : 51.5 deg-C
    Warning level.......: 77.0 deg-C
    Shutdown level......: 80.0 deg-C

My PSUs are both Rev A, if that helps. I'm also using a handful (8, I think offhand?) PoE powered devices.

Would it be worthwhile to rip the thing apart and put a new thermal pad on the CPU? I've looked, and the switch is spotless inside (I originally suspected dust, etc).

See below for my temps over the last 24h (ignore the bit where my Zabbix instance was down for maintenance). You can clearly see it bouncing between fan speed 1 and speed 2:

Screen Shot 2022-09-04 at 23.49.56.png

Thoughts?

EDIT: Just glanced, my R720XD two rack-Us away from this switch sits at 21C.
 
Last edited:

jasonwc

Member
Dec 31, 2018
49
18
8
Are there different revs of the ICX6450-24P?

Mine uses 35W in IDLE, @fohdeesha says 25W and another guy from a German forum says his one needs 20W in IDLE?
When you say idle, is that with no devices connected? Every attached copper port will use power even if the devices are idle, and all active transceivers will require power. These switches don’t have energy efficient Ethernet so they provide enough power for the full 100M reach over copper even if you have short runs.
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,333
810
113
When you say idle, is that with no devices connected? Every attached copper port will use power even if the devices are idle, and all active transceivers will require power. These switches don’t have energy efficient Ethernet so they provide enough power for the full 100M reach over copper even if you have short runs.
No cables attached, not even console. The other guy says that he gets 26W with 14 active RJ45 and 2 active SFP+ connections (but nearly no traffic)

Switchboard says Rev:03 GP (printed on the PCB)

Anyone else with a 6450 want to check their rev?
 
Last edited:

jasonwc

Member
Dec 31, 2018
49
18
8
No cables attached, not even console. The other guy says that he gets 26W with 14 active RJ45 and 2 active SFP+ connections (but nearly no traffic)

Switchboard says Rev:03 GP (printed on the PCB)

Anyone else with a 6450 want to check their rev?
There may be a difference in power consumption between the non-PoE and PoE models simply because power supplies tend to be very inefficient under 10% utilization. The 6450-24 has a 65W rating. The 6450-48 has a 100W rating. The 6450-24p is rare at 525W. The 6450-48p is rated at 880W. I would assume the PoE models would idle at a higher power consumption due to PSU inefficiency.

The spec sheets also seem to indicate there may be a difference running at 120V or 240V.

I have an ICX6450-48p currently not in use. I can plug it in to Kill-A-Watt to check the consumption on 120V. My ICX6610-48p is connected to my 240V UPS.