Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[bwahaha]

Pinging 192.168.1.50 with 32 bites of data
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.

I take it destination host unreachables not good

unreachable is good. Now plug the switch into the gateway, and try again.

 

[Tekgnome]

What port on the Brocade should be plugged into the gateway? I just have one of the rj45s on the front.

 

[Tekgnome]

I turned off WiFi on my gateway.

Unplugged my omada access points
The only devices that are plugged in are the laptop and the brocade.

The gateway is only showing 1 device connected (my laptop)

Edit

I used the other rj45 port on the back and connected it to the gateway and I get a green indicator light now.

Now in putty I get a new error message.

Received error message-- code 1

Edit 2

Looks like something to do with where my files are pointing or something.

Edit 3

*Facepalm* I completely missed the part under preparation that clearly says to connect the management ethernet port to my network and not to use any of the regular switch ports yet. Half my problems could have been solved if I just read the freaking instructions

 

[bwahaha]

Pinging 192.168.1.50 with 32 bites of data
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.
Reply from 192.168.1.137: destination host unreachable.

I take it destination host unreachables not good

Destination unreachable means there's no computer with that IP that your computer can find. We were looking to make sure the static ip on the switch wouldn't interfere with anything else on the network.

 

[Tekgnome]

Destination unreachable means there's no computer with that IP that your computer can find. We were looking to make sure the static ip on the switch wouldn't interfere with anything else on the network.

We are in business!

I finished flashing the bootloader and am working on the primary

Edit

Flashed the primary
Flashed the PoE module
Removed stacking
Set the serial number
Set sfp+ to 10g
Enabled PoE to my Omada AP's

 

[shpitz461]

Is there a way to set fan speed? Are the 6610 fans spinning higher than let's say a 7250 (from Terry Henry's videoes)?
SSH@ccx6610#dm fan:

FRU 1 Fan 1 Speed at 8333 RPM.
FRU 1 Fan 2 Speed at 9060 RPM.
FRU 2 Fan 1 Speed at 8132 RPM.
FRU 2 Fan 2 Speed at 9121 RPM.

SSH@ccx6610>show chassis

The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
Model Number: 23-0000142-02
Serial Number: 4KF
Firmware Ver: B
Power supply 1 Fan Air Flow Direction: Front to Back
Power supply 2 (AC - PoE) present, status ok
Model Number: 23-0000142-02
Serial Number: 5SH
Firmware Ver: B
Power supply 2 Fan Air Flow Direction: Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 55.5 deg-C

Fan speed switching temperature thresholds:
Speed 1: NM<----->84 deg-C
Speed 2: 79<-----> 87 deg-C (shutdown)

Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back
MAC 1 Temperature Readings:
Current temperature : 39.0 deg-C
MAC 2 Temperature Readings:
Current temperature : 45.5 deg-C
CPU Temperature Readings:
Current temperature : 55.5 deg-C
sensor A Temperature Readings:
Current temperature : 44.5 deg-C
sensor B Temperature Readings:
Current temperature : 44.5 deg-C
sensor C Temperature Readings:
Current temperature : 33.5 deg-C
stacking card Temperature Readings:
Current temperature : 50.5 deg-C
Warning level.......: 77.0 deg-C
Shutdown level......: 87.0 deg-C

 

[Tekgnome]

Is there a way to set fan speed? Are the 6610 fans spinning higher than let's say a 7250 (from Terry Henry's videoes)?
SSH@ccx6610#dm fan:


SSH@ccx6610>show chassis

What rev power supplies do you have?

From my understanding rev A units are louder than rev B and C.

I'm not at my switch right now but I can check mine later. From my experience with mine I can tell you that aside from my passively cooled mikrotik it's the quietest piece of networking equipment I own. It's even quieter than my Plex server. That being said, it's not inaudible, it does have a quiet low hum to it.

 

[blunden]

Do you guys have any suggestions for powering a GWN7664 AP with its 2.5Gbe port? My ICX 6610 only provides PoE over 1Gbe connections.
Should I just get a 2.5Gbe PoE injector and connect it to one of the 10Gbe ports? Will that even work with SFP+ adapters?
Or should I get a 6-8 port 2.5Gbe with PoE+ and 1x10Gbe SFP+ and connect it to the 6610? Does it have to be managed in order to keep VLAN information?
Maybe something like this: https://www.aliexpress.us/item/3256804636026785.html
Or maybe this one? https://www.amazon.com/Davuaz-IEEE802-3af-Compliant-Fanlesss-Unmanaged/dp/B0C862NJW3/ref=sr_1_10
Thanks!

I would strongly suspect that your switch is too old to have native support for 2.5 and 5 GbE so if you are using the 10 GbE SFP+ ports you need a module that negotiates 10 GbE to the switch but 2.5 GbE to the device. Those exist, but most of them don't work properly based on information on this forum. The only ones that apparently work properly in bidirectional tests are ones using Aquantia chips. Thankfully there is a module from 10Gtek that uses the AQR113C chip that is reasonably priced, specifically the 10Gtek ASF-10G2-T (note the 2 in the model name). Note that it requires Flow Control to be enabled since it uses PAUSE frames to limit the speed. If you search around, you can find threads about these on here.

Some unmanaged switches pass VLAN information, some don't. I don't know about those Realtek based switches in particular but there are several owners of those switches on these forums so you could ask in one of the existing threads for those switches. If they do pass VLAN information that's certainly also an option instead of the SFP+ module.

 

[shpitz461]

What rev power supplies do you have?

From my understanding rev A units are louder than rev B and C.

I'm not at my switch right now but I can check mine later. From my experience with mine I can tell you that aside from my passively cooled mikrotik it's the quietest piece of networking equipment I own. It's even quieter than my Plex server. That being said, it's not inaudible, it does have a quiet low hum to it.

Both PSU's are ' Firmware Ver: B'.
Do you have PoE turned on?
I think without PoE the switch is farily quiet, but if I recall correctly, when I initially set it up and turned on PoE, it became much much louder ever since.

 

[Tekgnome]

Both PSU's are ' Firmware Ver: B'.
Do you have PoE turned on?
I think without PoE the switch is farily quiet, but if I recall correctly, when I initially set it up and turned on PoE, it became much much louder ever since.

I'm running PoE on 6 ports. I have a Omada Access Point, a reolink doorbell and 4 turret cams. I'll probably expand that to 8 turret cams by December, I plan to get a Poe splitter to power my security system as well as a second AP to run in my garage.

So 6 right now and I'll end up with around 12 PoE devices.

 

[shpitz461]

Can you run the fan command to see what fan speeds yours are running @?
I only have two PoE ports being used, one for AP (port 1/1/2), one for a turret camera (port 1/1/6):

SSH@ccx6610#show inline power

Power Capacity: Total is 1496000 mWatts. Current Free is 1436000 mWatts.

Power Allocations: Requests Honored 35 times


Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------
1/1/1 On Off 0 0 n/a n/a 3 n/a
1/1/2 On On 9700 30000 802.3at Class 4 1 n/a
1/1/3 On Off 0 0 n/a n/a 3 n/a
1/1/4 On Off 0 0 n/a n/a 3 n/a
1/1/5 On Off 0 0 n/a n/a 3 n/a
1/1/6 On On 14300 30000 802.3at Class 4 3 n/a
1/1/7 On Off 0 0 n/a n/a 3 n/a
1/1/8 On Off 0 0 n/a n/a 3 n/a
1/1/9 Off Off 0 0 n/a n/a 3 n/a
1/1/10 Off Off 0 0 n/a n/a 3 n/a
1/1/11 Off Off 0 0 n/a n/a 3 n/a
1/1/12 Off Off 0 0 n/a n/a 3 n/a
1/1/13 Off Off 0 0 n/a n/a 3 n/a
1/1/14 Off Off 0 0 n/a n/a 3 n/a

 

[Tekgnome]

Can you run the fan command to see what fan speeds yours are running @?
I only have two PoE ports being used, one for AP (port 1/1/2), one for a turret camera (port 1/1/6):

Spoiler: dm fan

FRU 1 Fan 1 Speed at 5793 RPM.
FRU 1 Fan 2 Speed at 6553 RPM.
FRU 2 Fan 1 Speed at 5844 RPM.
FRU 2 Fan 2 Speed at 6192 RPM.

Spoiler: show chassis

The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
Model Number: 23-0000142-02
Serial Number: 9FK
Firmware Ver: A
Power supply 1 Fan Air Flow Direction: Front to Back
Power supply 2 (AC - PoE) present, status ok
Model Number: 23-0000142-02
Serial Number: 91G
Firmware Ver: A
Power supply 2 Fan Air Flow Direction: Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature: 61.5 deg-C

Fan speed switching temperature thresholds:
Speed 1: NM<----->76 deg-C
Speed 2: 71<-----> 80 deg-C (shutdown)

Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back
MAC 1 Temperature Readings:
Current temperature : 49.0 deg-C
MAC 2 Temperature Readings:
Current temperature : 56.5 deg-C
CPU Temperature Readings:
Current temperature : 61.5 deg-C
sensor A Temperature Readings:
Current temperature : 50.5 deg-C
sensor B Temperature Readings:
Current temperature : 53.5 deg-C
sensor C Temperature Readings:
Current temperature : 39.5 deg-C
stacking card Temperature Readings:
Current temperature : 53.5 deg-C
Warning level.......: 77.0 deg-C
Shutdown level......: 80.0 deg-C

 

[shpitz461]

So yours is a Rev A, and they run @ much lower rpm than mine.
@fohdeesha and everyone else, any idea why my fans are spinning much faster?
These are the fans that came in my 6610:

 

[Tekgnome]

So yours is a Rev A, and they run @ much lower rpm than mine.
@fohdeesha and everyone else, any idea why my fans are spinning much faster?
These are the fans that came in my 6610:
View attachment 32477

That's interesting, I was under the impression that the Rev B and C were quieter. While definitely audible I wouldn't consider my switch loud. I'm in the process of installing a cabinet and once it's enclosed it will probably be completely silent. Currently mine is in an open air network rack.

The current sound profile sounds kinda like some running water behind a wall. My 160tb Epyc Plex server easily drowns it out.

 

[itronin]

That's interesting, I was under the impression that the Rev B and C were quieter.

check the label on the psu. All of my audible comparisons were based on REV A B C as listed on the psu label. My A's were always louder than the B's and C that I have.

 

[shpitz461]

Both of my PSU's are Rev B and compared to @Tekgnome mine are set to ~9k RPM while his are ~6k RPM.
Mine:

FRU 1 Fan 1 Speed at 8333 RPM.
FRU 1 Fan 2 Speed at 9060 RPM.
FRU 2 Fan 1 Speed at 8132 RPM.
FRU 2 Fan 2 Speed at 9121 RPM.

@Tekgnome's:

FRU 1 Fan 1 Speed at 5793 RPM.
FRU 1 Fan 2 Speed at 6553 RPM.
FRU 2 Fan 1 Speed at 5844 RPM.
FRU 2 Fan 2 Speed at 6192 RPM.

Is there a way to set these levels?

And it's funny, right next to the switch (got it laying against a wall by the floor) is my Epyc server as well, and the switch is WAY louder...

 

[shpitz461]

Another issue:
I tried turning on https, following Terry's video here:

Generated a certificate and enabled https in the switch, but this is what I get in Chrome:


Am I missing something?

 

[shpitz461]

I would strongly suspect that your switch is too old to have native support for 2.5 and 5 GbE so if you are using the 10 GbE SFP+ ports you need a module that negotiates 10 GbE to the switch but 2.5 GbE to the device.

Yep, the 6610 is old and doesn't have 2.5GbE PoE, only 1GbE.
I actually have one of my boys connected to one of the 10GbE ports through a MoCA adapter that does 2.5GbE, and it works great, so I know the hardware I have is capable.
I would love to get my hands on a ICX7150-48ZP but these go for $1500+ on eBay right now
Got the 6610 a while ago for $130.

 

[Tekgnome]

Yep, the 6610 is old and doesn't have 2.5GbE PoE, only 1GbE.
I actually have one of my boys connected to one of the 10GbE ports through a MoCA adapter that does 2.5GbE, and it works great, so I know the hardware I have is capable.
I would love to get my hands on a ICX7150-48ZP but these go for $1500+ on eBay right now
Got the 6610 a while ago for $130.

Oh those ICX7150-48ZP look great, I can't wait for 10 years to pass so I can pick one up for $130 on eBay

 

[shpitz461]

Haha, I rebooted the switch by mistake after trying to get it to sync to NTP (which doesn't work for me), and now the fans show the same speeds as reported by others previously:

SSH@ccx6610#dm fan
FRU 1 Fan 1 Speed at 5869 RPM.
FRU 1 Fan 2 Speed at 6250 RPM.
FRU 2 Fan 1 Speed at 5720 RPM.
FRU 2 Fan 2 Speed at 6521 RPM.

Here's my NTP status:

show ntp status
Clock is unsynchronized, no reference clock
NTP server mode is enabled, NTP client mode is enabled
NTP master mode is disabled, NTP master stratum is 8
NTP is not in panic mode

Here's my NTP associations:

show ntp ass
address ref clock st when poll reach delay offset disp
~96.126.122.39 INIT 16 - 256 0 0.00 0.000 15937.
~4.2.2.2 INIT 16 - 256 0 0.00 0.000 15937.
* synced, # selected, + candidate, - outlayer, x falseticker, ~ configured

I have DNS configured, but looks like something is still not right when I ping, I do get IP resolution but it says request timeout:

SSH@ccx6610#ping
Incomplete command.
SSH@ccx6610#ping google.com

Type Control-c to abort
Sending DNS Query to 192.168.1.251
Sending 1, 16-byte ICMP Echo to 142.250.9.101, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.
SSH@ccx6610#ping us.pool.ntp.org

Type Control-c to abort
Sending DNS Query to 192.168.1.251
Sending 1, 16-byte ICMP Echo to 184.105.182.16, timeout 5000 msec, TTL 64
Type Control-c to abort
Request timed out.
No reply from remote host.

Any idea what I'm missing?
Maybe I'm missing gateway? Is this how I configure the gateway?
#ip default-network 192.168.1.254/24
Do i need to be at the 've 1' level? or @ the config root?
Thanks