Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

thehedgefrog

New Member
Sep 14, 2022
8
1
3
So I am trying to make my Sonos speakers work and since I switched to the ICX6610 I just can't get it to work.

I disabled spanning-tree on relevant ports, setup IGMP snooping, put an ip helper-address... I have PIMD setup on pfSense as per the old working config with an interface to vlan 30 (IoT) and the interface going to my switch. 192.168.200.1 is pfSense, 192.168.200.2 is the switch.

Here is my config:
Code:
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  no legacy-inline-power
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 20 name trusted by port
tagged ethe 1/1/43 to 1/1/45 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/25 to 1/1/30 ethe 1/3/5 to 1/3/6
router-interface ve 20
multicast passive
!
vlan 30 name iot by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/33 to 1/1/40
!
vlan 50 name security by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/31 to 1/1/32
!
vlan 70 name GUEST by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
!
vlan 200 name lab by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/1 to 1/1/14 ethe 1/3/1 to 1/3/4
router-interface ve 200
!
vlan 220 name WFH by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
!
vlan 255 name management by port
tagged ethe 1/1/43 to 1/1/45 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/15 to 1/1/24
!
!
!
!
!
optical-monitor
aaa authentication web-server default local
aaa authentication login default local
enable telnet authentication
enable telnet password .....
bpdu-flood-enable
hostname core-switch
ip dhcp-client disable
ip dns server-address 10.200.0.2 10.20.0.1
ip route 0.0.0.0/0 192.168.200.1
ip add-host-route-first
ip multicast passive
!
username mcct password .....
!
!
clock summer-time
clock timezone gmt GMT-05
!
!
ntp
server 162.159.200.123
server 216.239.35.0
!
!
web-management https
!
router ospf
area 0
!
!
!
!
!
!
!
interface ethernet 1/1/43
dual-mode  255
no spanning-tree
inline power
!
interface ethernet 1/1/44
dual-mode  255
no spanning-tree
inline power
!
interface ethernet 1/1/45
dual-mode
!
interface ethernet 1/1/47
dual-mode
!
interface ethernet 1/1/48
dual-mode
no spanning-tree
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.200.2 255.255.255.0
ip ospf area 0
!
interface ve 20
ip address 10.20.1.1 255.255.0.0
ip helper-address 1 10.200.0.2
ip helper-address 2 192.168.200.1
ip ospf area 0
!
interface ve 200
ip address 10.200.1.1 255.255.0.0
ip ospf area 0
!
!
!
!
!
!
!
!
!
end
Here are the show multicast results for vlan 20 (trusted) and vlan 30 (iot)
Code:
SSH@core-switch(config)#sh ip mu vlan 30
Version=2, Intervals: Query=125, Group Age=260, Max Resp=10, Other Qr=255

VL30: dft V2, glb cfg passive, 3 grp, 2 (*G) cache, rtr ports,
    router ports: e1/1/48(260) 10.30.0.1,

  e1/1/39   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/40   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/43   has    3 groups,
This interface is non-Querier (passive)
default V2
    group: 239.255.3.22, life = 260
    group: 239.255.255.251, life = 240
    group: 239.255.255.250, life = 240

  e1/1/44   has    1 groups,
This interface is non-Querier (passive)
default V2
    group: 239.255.255.250, life = 240

  e1/1/48   has    1 groups,
This interface is non-Querier (passive)
default V2
  **** Warning! has V3 (age=80)  nbrs
    group: 239.255.255.250, life = 240
   
   
SSH@core-switch(config)#sh ip mu vlan 20
Version=2, Intervals: Query=125, Group Age=260, Max Resp=10, Other Qr=255

VL20: dft V2, vlan cfg passive, 0 grp, 1 (*G) cache, no rtr port,

  e1/1/25   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/43   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/44   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/48   has    0 groups,
This interface is non-Querier (passive)
default V2
 

kevindd992002

Member
Oct 4, 2021
136
9
18
@fohdeesha so I'm following your guides in updating the device firmware, bootloader, and poe firmware of my ICX6450. I'm already at the poe firmware part and this is what I got:

Code:
ICX6450-48P Router#inline power install-firmware stack-unit 1 tftp 192.168.20.21                                                                                                                                                                                                                                              ICX64xx/icx64xx_poeplus_02.1.0.b004.fw
ICX6450-48P Router#Flash Memory Write (8192 bytes per dot) ..........
tftp download successful file name = poe-fw
Sending PoE Firmware to Unit 1.
Firmware version from File: 2.1.1
PoE Warning: Upgrading firmware in slot 1....DO NOT HOTSWAP OR POWER DOWN THE MO                                                                                                                                                                                                                                             DULE.
PoE Info: FW Download on slot 1...sending download command...
PoE Info: FW Download on slot 1...TPE response received.
PoE Info: FW Download on slot 1...sending erase command...
PoE Info: FW Download on slot 1...erase command...accepted.
PoE Info: FW Download on slot 1...erasing firmware memory...
PoE Info: FW Download on slot 1...erasing firmware memory...completed
PoE Info: FW Download on slot 1...sending program command...
PoE Info: FW Download on slot 1...sending program command...accepted.
PoE Info: FW Download on slot 1...programming firmware...takes around 12 minutes                                                                                                                                                                                                                                             ....
U1-MSG: PoE Info: Firmware Download on slot 1.....10 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....20 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....30 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....40 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....50 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....60 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....70 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....80 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....90 percent completed.
U1-MSG: PoE Info: Firmware Download on slot 1.....100 percent completed.
PoE Info: FW Download on slot 1...programming firmware...completed.
PoE Info: FW Download on slot 1...upgrading firmware...completed. Module will be                                                                                                                                                                                                                                              reset.
PoE Info: Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Info: Programming Ruckus defaults. Step 1: Writing port defaults on module i                                                                                                                                                                                                                                             n slot 1....
PoE Info: Programming Ruckus Defaults: Step 2: Writing PM defaults on module in                                                                                                                                                                                                                                              slot 1.
PoE Info: Programming Ruckus defaults. Step 3: Writing user byte 0xf3 on module                                                                                                                                                                                                                                              in slot 1.
PoE Info: Programming Ruckus defaults. Step 4: Saving settings on module in slot                                                                                                                                                                                                                                              1.
PoE Info: Programming Ruckus defaults....completed.
PoE Info: PoE module 1 of Unit 1 initialization is done.

ICX6450-48P Router#write memory
ICX6450-48P Router#reload
Sent SIGTERM to all processesn'):
Sent SIGKILL to all processes
Requesting system reboot
Restarting system.
Questions:

1. Are those poe errors expected? It's worth noting that I don't see the same errors when booting (the poe modules are initialized just fine). So these two errors only ever showed during the poe firmware update.

Bootloader Version: 10.1.05T310 (Mar 19 2015 - 16:39:59)


Model ID: 1.0.0.1.1.0

Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
## Booting image at 00007fc0 ...
Created: 2020-04-23 17:58:12 UTC
Data Size: 9870536 Bytes = 9.4 MB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
OK

Starting kernel in BE mode ...
Uncompressing Image................................................................................................................................................................................ ................................................................................................................................................................................................... ........................... done, booting the kernel.
Config partition mounted.
Creating TUN device
Starting the FastIron.
FIPS Disabled:pORT NOT DISABLED
platform type 47
OS>Unable to set the kernel wall time
Starting Main Task .CPSS DxCh Version: cpss3.4p1 release
Pre Parsing Config Data ...

Parsing Config Data ...
Enable optical monitoring and set alarm/warn interval to default(3 minutes)

System initialization completed...console going online.
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on Apr 23 2020 at 10:57:06 labeled as ICX64R08030u
(9871112 bytes) from Primary ICX64R08030u.bin
SW: Version 08.0.30uT313
Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
HW: Stackable ICX6450-48-HPOE
==========================================================================
UNIT 1: SL 1: ICX6450-48P POE 48-port Management Module
Serial #: 2ax5o2jk68e
License: ICX6450_PREM_ROUTER_SOFT_PACKAGE (LID: H4CKTH3PLN8)
P-ENGINE 0: type DEF0, rev 01
P-ENGINE 1: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
512 MB DRAM
STACKID 1 system uptime is 28 second(s)
The system started at 08:00:38 GMT+08 Thu Jan 01 1970

The system : started=warm start reloaded=by "reload"

ICX6450Switch>
Stack unit 1 PS 1, Internal Power supply detected and up.

Stack unit 1 PS 1, Internal Power supply detected and up.
PoE: Stack unit 1 PS 1, Internal Power supply with 740000 mwatts capacity is up
PoE Info: Adding new 54V capacity of 740000 mW, total capacity is 740000, total free capacity is 740000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
PoE Info: PoE module 1 of Unit 1 initialization is done.

2. In your guide you mentioned

"#after a few seconds, hit enter to return to cli".

I did this after I saw this message "PoE Info: PoE module 1 of Unit 1 initialization is done." Is that fine?

3. In your guide, you mentioned:

#you'll probably get a message that it hasn't finished. it can take up to 10 minutes
#run "show log" occasionally to monitor the update progress
#try the "reload" command again once it's reached 100%

I didn't see this message. Is this because I hit enter when the poe firmware update process is already done?

4. What's the difference between reload and reset? I see that they both reboot the switch anyway.
Does anyone have any ideas on this please?
 

Vesalius

Active Member
Nov 25, 2019
263
205
43
Try this post: https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-345446

PIM Dense
So I am trying to make my Sonos speakers work and since I switched to the ICX6610 I just can't get it to work.

I disabled spanning-tree on relevant ports, setup IGMP snooping, put an ip helper-address... I have PIMD setup on pfSense as per the old working config with an interface to vlan 30 (IoT) and the interface going to my switch. 192.168.200.1 is pfSense, 192.168.200.2 is the switch.

Here is my config:
Code:
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  no legacy-inline-power
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 20 name trusted by port
tagged ethe 1/1/43 to 1/1/45 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/25 to 1/1/30 ethe 1/3/5 to 1/3/6
router-interface ve 20
multicast passive
!
vlan 30 name iot by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/33 to 1/1/40
!
vlan 50 name security by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/31 to 1/1/32
!
vlan 70 name GUEST by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
!
vlan 200 name lab by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/1 to 1/1/14 ethe 1/3/1 to 1/3/4
router-interface ve 200
!
vlan 220 name WFH by port
tagged ethe 1/1/43 to 1/1/44 ethe 1/1/47 to 1/1/48
!
vlan 255 name management by port
tagged ethe 1/1/43 to 1/1/45 ethe 1/1/47 to 1/1/48
untagged ethe 1/1/15 to 1/1/24
!
!
!
!
!
optical-monitor
aaa authentication web-server default local
aaa authentication login default local
enable telnet authentication
enable telnet password .....
bpdu-flood-enable
hostname core-switch
ip dhcp-client disable
ip dns server-address 10.200.0.2 10.20.0.1
ip route 0.0.0.0/0 192.168.200.1
ip add-host-route-first
ip multicast passive
!
username mcct password .....
!
!
clock summer-time
clock timezone gmt GMT-05
!
!
ntp
server 162.159.200.123
server 216.239.35.0
!
!
web-management https
!
router ospf
area 0
!
!
!
!
!
!
!
interface ethernet 1/1/43
dual-mode  255
no spanning-tree
inline power
!
interface ethernet 1/1/44
dual-mode  255
no spanning-tree
inline power
!
interface ethernet 1/1/45
dual-mode
!
interface ethernet 1/1/47
dual-mode
!
interface ethernet 1/1/48
dual-mode
no spanning-tree
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.200.2 255.255.255.0
ip ospf area 0
!
interface ve 20
ip address 10.20.1.1 255.255.0.0
ip helper-address 1 10.200.0.2
ip helper-address 2 192.168.200.1
ip ospf area 0
!
interface ve 200
ip address 10.200.1.1 255.255.0.0
ip ospf area 0
!
!
!
!
!
!
!
!
!
end
Here are the show multicast results for vlan 20 (trusted) and vlan 30 (iot)
Code:
SSH@core-switch(config)#sh ip mu vlan 30
Version=2, Intervals: Query=125, Group Age=260, Max Resp=10, Other Qr=255

VL30: dft V2, glb cfg passive, 3 grp, 2 (*G) cache, rtr ports,
    router ports: e1/1/48(260) 10.30.0.1,

  e1/1/39   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/40   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/43   has    3 groups,
This interface is non-Querier (passive)
default V2
    group: 239.255.3.22, life = 260
    group: 239.255.255.251, life = 240
    group: 239.255.255.250, life = 240

  e1/1/44   has    1 groups,
This interface is non-Querier (passive)
default V2
    group: 239.255.255.250, life = 240

  e1/1/48   has    1 groups,
This interface is non-Querier (passive)
default V2
  **** Warning! has V3 (age=80)  nbrs
    group: 239.255.255.250, life = 240
  
  
SSH@core-switch(config)#sh ip mu vlan 20
Version=2, Intervals: Query=125, Group Age=260, Max Resp=10, Other Qr=255

VL20: dft V2, vlan cfg passive, 0 grp, 1 (*G) cache, no rtr port,

  e1/1/25   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/43   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/44   has    0 groups,
This interface is non-Querier (passive)
default V2

  e1/1/48   has    0 groups,
This interface is non-Querier (passive)
default V2
 

baskethammer

New Member
Nov 4, 2022
6
1
3
Resolved: I have a ve on vlan1. Thanks to @kpfleming 's note, i tried to route-only the port and got a helpful error message:
Code:
SSH@chonk(config-if-e1000-1/1/48)#no route-only
Error - Cannot change Route-Only setting on physical port that is part of a VE
Note - Route-Only is not active (config is ignored) when the physical port is part of a VE
Removed the ve for vlan1 and can assign ip address to port in line with documentation, but it breaks my network (nothing left active on vlan1, remove ve1 and kaboom everything goes haywire). In any case (after rebooting to saved config):

Code:
SSH@chonk(config)#vlan 999
SSH@chonk(config-vlan-999)#untagged eth 1/1/48    
Added untagged port(s) ethe 1/1/48 to port-vlan 999.
SSH@chonk(config-if-e1000-1/1/48)#ip 
  access-group         Apply access control list
  address              Assign IP address to this interface
.....
all appears to be well and i can assign ip address..
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
I am going crazy trying to get this to work !

Guys, experienced in VMware and Cisco - new to Brocade but think i understand the basic now.

Purchased 2 x 6610 one with POE and one without - followed the excellent docs by fodeesha and got them both running and licensed.

I want to use them in my home lab to replace a Cisco 4948 and numerous small dumb switches.

The plan was to breakout the 1/2/x ports on the back and put them to my ESXi (7.03) hosts to give them 10GB capability

I an in the process of revamping my home lab from using HP/Compaq Elite 8300 boxes (3) on ESXI 6.7 with quad port intel Gb cards to Dell Optiplex 7050 (3) with 10GB Intel DA-2 adapters.

I also have a number of Intel 540T (Dual port) adapters and some sinlge port DA-1 adapters.

I have purchase the FS.com QSFP to SFP+ breakouts (3 of them) as well as Some Dell cables and finally some Arista cables.

My baseline issue is that i can not get the 2nd port to work reliably on either the 540T (with Dell transceivers) into the 1/3/x slots or the 520 into the 1/2/x cables.

I have tried swapping out cables thinking i had some duds or interoperability issues but this has not changed the problem - i have upgrade and downgraded the Intel drivers on ESXi and no change

I have now stood up the POE 6610 in an isolated environment with a spare host and some new Arista cables that arrived today and exactly the same thing happens - the ports show as active and show in ESXi as connected and then they just move into a down state and nothing i do seems to be able to revive them - the only solution i have found so far is to shutdown the host and bring it back up

1667794009186.png

and the other port which is operating fine


1667794139932.png

I am about to replace the dual port card with 2 x single port and see if i can see any difference there

Does ANYONE have any ideas about what else i can try ?

Craig
 

LodeRunner

Active Member
Apr 27, 2019
557
238
43
What's the correct way of shutting down these ICX range?
Same as pretty much any other switch: pull power. The running OS is loaded into RAM at boot; storage is only touched at boot time or when the config is committed (wr me).
 

Damo

Active Member
Sep 7, 2022
131
37
28
Same as pretty much any other switch: pull power. The running OS is loaded into RAM at boot; storage is only touched at boot time or when the config is committed (wr me).
Ok was hoping there was a shutdown command. I got a cheap 6450 that wont boot after a few shutdowns was thinking I caused it by pulling the plug.
 

kevindd992002

Member
Oct 4, 2021
136
9
18
Anyone with the ICX6450 that had the same PoE error only during the PoE firmware upgrade?

PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
Anyone with the ICX6450 that had the same PoE error only during the PoE firmware upgrade?

PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Nope but i have 3 of them to do that have just arrived from the USA - will try and get onto them this week and advise

Craig
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
COLD START on 6610

Guys, I have been battling with a 6610 POE for a while and have noticed that at random times - not load related - and the switch is dual PS on a UPS that i get a COLD START message in the logs and also sent to the SYSLOG server i have setup - does this mean the switch is randomly rebooting - or is this some other innoccuous message ?

Craig
 

kpfleming

Active Member
Dec 28, 2021
459
237
43
Pelham NY USA
Removed the ve for vlan1 and can assign ip address to port in line with documentation, but it breaks my network (nothing left active on vlan1, remove ve1 and kaboom everything goes haywire).
Yep, I ran into this recently via a roundabout path (removed the untagged VLAN from a route-only port which had an IP address so it reverted back to VLAN 1 and then boom...).

If there is a route-only port on a VLAN, it appears that no other ports on that VLAN are usable, even though the software won't stop you from configuring it that way. Thus, I've got all of my route-only ports setup on their own individual VLANs, one port each.
 
  • Like
Reactions: baskethammer

itronin

Well-Known Member
Nov 24, 2018
1,354
899
113
Denver, Colorado
COLD START on 6610

Guys, I have been battling with a 6610 POE for a while and have noticed that at random times - not load related - and the switch is dual PS on a UPS that i get a COLD START message in the logs and also sent to the SYSLOG server i have setup - does this mean the switch is randomly rebooting - or is this some other innoccuous message ?

Craig
I have seen this behavior on simulated & non sine wave UPS's... ie. low cost UPS's. The ICX 6610 psu is pretty sensitive to power. Made no difference running a single or dual PSU. Made no difference rev A or rev B or a mix of rev PSU's. I have seen the switch stay up for 1 or 2 days or up to 20+ days but eventually it gets unhappy on the low cost UPS's and randomly restarts.

Moved the 6610's to an EATON 2200 and the cold start/random reboots disappeared. They also disappeared just running them on street power. I put them back on a cheap APC or Cyberpower UPS and they periodically reboot.
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
I have seen this behavior on simulated & non sine wave UPS's... ie. low cost UPS's. The ICX 6610 psu is pretty sensitive to power. Made no difference running a single or dual PSU. Made no difference rev A or rev B or a mix of rev PSU's. I have seen the switch stay up for 1 or 2 days or up to 20+ days but eventually it gets unhappy on the low cost UPS's and randomly restarts.

Moved the 6610's to an EATON 2200 and the cold start/random reboots disappeared. They also disappeared just running them on street power. I put them back on a cheap APC or Cyberpower UPS and they periodically reboot.
Thanks for the quick answer - not a cheap UPS unfortunately - it is an APC 3000RMT XL - double conversion pure sine wave - running from a Lithium battery bank good for approx 8 hours run time

So does COLD start actually mean a full switch reboot ? i.e. loss of power

Craig
 

itronin

Well-Known Member
Nov 24, 2018
1,354
899
113
Denver, Colorado
Thanks for the quick answer - not a cheap UPS unfortunately - it is an APC 3000RMT XL - double conversion pure sine wave - running from a Lithium battery bank good for approx 8 hours run time

So does COLD start actually mean a full switch reboot ? i.e. loss of power

Craig
based on what I've seen its the same as pulling the cables and plugging them back in or hitting the kill switch on a pdu. FWIW when it first started I was thinking I had a bad switch so I pointed syslog to a server (same as you) to watch what was getting logged. going back into the switch and looking at show tech (I think) there was some crash dump information.

The only thing I can suggest is try running on street power (if yours is stable) for a while and see what happens.

My eaton is SLA and I would not expect that to make a difference though.

If you have access to an older APC 2000/3000 with SLA you might try that to rule it out though as I have seen different behaviors between newer APC's vs. some of the older "tanks". obviously YMMV and I have 0 experience with the newer Li-ION UPS's.

edit

you could also have bad switches I guess. You didn't buy them from g-electronic here in the states did you?
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
based on what I've seen its the same as pulling the cables and plugging them back in or hitting the kill switch on a pdu. FWIW when it first started I was thinking I had a bad switch so I pointed syslog to a server (same as you) to watch what was getting logged. going back into the switch and looking at show tech (I think) there was some crash dump information.

The only thing I can suggest is try running on street power (if yours is stable) for a while and see what happens.

My eaton is SLA and I would not expect that to make a difference though.

If you have access to an older APC 2000/3000 with SLA you might try that to rule it out though as I have seen different behaviors between newer APC's vs. some of the older "tanks". obviously YMMV and I have 0 experience with the newer Li-ION UPS's.

edit

you could also have bad switches I guess. You didn't buy them from g-electronic here in the states did you?
Yeah i might move it back to street power for a little while and see what happens - hopefully it is not that finicky with power requirements !!

No i didn't get them from g-electronic - but it was from a US Ebay vendor and they were given a bashing in transit - so i ended up getting them for free through the EBAY global shipping program - thought it was OK until i see these issues - maybe not

Will report back in a couple of days

Craig
 

itronin

Well-Known Member
Nov 24, 2018
1,354
899
113
Denver, Colorado
Yeah i might move it back to street power for a little while and see what happens - hopefully it is not that finicky with power requirements !!

No i didn't get them from g-electronic - but it was from a US Ebay vendor and they were given a bashing in transit - so i ended up getting them for free through the EBAY global shipping program - thought it was OK until i see these issues - maybe not

Will report back in a couple of days

Craig
one thing I never tried... though easy enough to do in a couple of weeks when I have time. run one PSU on a cheap UPS and another on streeet power. see if it is more stable. problem is that it could stay up for like 3 weeks on the cheap UPS before cycling.

btw, I should add I did NOT always see a crash report after it did the spontaneous reboot.
 

Craig Curtin

Member
Jun 18, 2017
103
20
18
60
one thing I never tried... though easy enough to do in a couple of weeks when I have time. run one PSU on a cheap UPS and another on streeet power. see if it is more stable. problem is that it could stay up for like 3 weeks on the cheap UPS before cycling.

btw, I should add I did NOT always see a crash report after it did the spontaneous reboot.

OK got me thinking after your posts (BTW - thanks again) - here is a dump of some of the show tech - should i be concerned in here - is this indicating an ECC memory issues ?

==========================================================================
==========================================================================
BEGIN : show sysmon counters all
CONTEXT : SSH#1 : SYSTEM MONITORING INFO
TIME STAMP : Nov 8 12:41:50.418
HW/SW INFO : ICX6610-48-HPOE/FCXR08030u
==========================================================================
Sysmon error detected on: Stacking Unit 1 (number of times)

****Stacking unit 1 (ICX) Link error detect
1st device, core 2. port 42
Link error detect = 0 remote fault detect = 0 gearbox error = 0
1st device, core 3. port 58
Link error detect = 0 remote fault detect = 0 gearbox error = 0
2nd device, core 0. port 10
Link error detect = 0 remote fault detect = 0 gearbox error = 0
2nd device, core 1. port 26
Link error detect = 0 remote fault detect = 0 gearbox error = 0

==========================
Sysmon ECC error detected on: Stacking Unit 1 (number of times)

****Stacking unit 1 (ICX) ecc error detect
ECC one-time error detect = 0 ECC two-time error detect = 0
==========================
==========================================================================
TIME STAMP : Nov 8 12:41:50.421
END : show sysmon counters all
TIME TAKEN : 120208 ticks (2404160 nsec)
==========================================================================

And does this unit think it is still part of a stack ??

==========================================================================
BEGIN : show stack
CONTEXT : SSH#1 : STACK DETAILS
TIME STAMP : Nov 8 12:41:50.664
HW/SW INFO : ICX6610-48-HPOE/FCXR08030u
==========================================================================

Stack Details
=============
T=11h6m39.2: alone: standalone, D: dynamic cfg, S: static
ID Type Role Mac Address Pri State Comment
1 S ICX6610-48P alone cc4e.24b9.c6d0 0 local None:0


+---+
2/1| 1 |2/6
+---+
Current stack management MAC is cc4e.24b9.c6d0

Image-Auto-Copy is Enabled.

Stack Port Status Neighbors
Unit# Stack-port1 Stack-port2 Stack-port1 Stack-port2
1 none none none none
==========================================================================
TIME STAMP : Nov 8 12:41:50.667
END : show stack
TIME TAKEN : 155765 ticks (3115300 nsec)
==========================================================================

Craig
 

kevindd992002

Member
Oct 4, 2021
136
9
18
Nope but i have 3 of them to do that have just arrived from the USA - will try and get onto them this week and advise

Craig
Thanks. FWIW, I reflashed the PoE controller firmware and this time I pressed enter to go back to the console after a few seconds (just like instructed in fohdeesha's guide) and I did not see the "PoE info" messages (which I assume are console messages) I saw earlier when I did not press enter (https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-353942)

The show log command only shows the " U1-MSG: PoE Info" messages which is why most people here probably didn't see anything related to this. When you upgrade the PoE controller firmware on your units, would you mind not pressing enter and waiting for the whole process to finish just like I did?
 

itronin

Well-Known Member
Nov 24, 2018
1,354
899
113
Denver, Colorado
And does this unit think it is still part of a stack ??

==========================================================================
BEGIN : show stack
CONTEXT : SSH#1 : STACK DETAILS
TIME STAMP : Nov 8 12:41:50.664
HW/SW INFO : ICX6610-48-HPOE/FCXR08030u
==========================================================================

Stack Details
=============
T=11h6m39.2: alone: standalone, D: dynamic cfg, S: static
ID Type Role Mac Address Pri State Comment
1 S ICX6610-48P alone cc4e.24b9.c6d0 0 local None:0


+---+
2/1| 1 |2/6
+---+
Current stack management MAC is cc4e.24b9.c6d0

Image-Auto-Copy is Enabled.

Stack Port Status Neighbors
Unit# Stack-port1 Stack-port2 Stack-port1 Stack-port2
1 none none none none
==========================================================================
TIME STAMP : Nov 8 12:41:50.667
END : show stack
TIME TAKEN : 155765 ticks (3115300 nsec)
==========================================================================

Craig
sooo the symptoms you described in an earlier post - about going crazy - were indicative to me of what I experienced not quite disabling stacking and trying to use one of the 40/10 breakouts. One worked for breakout - the other did not...

here's what my standalone switch shows and see the difference?

edit - just realized you cut n pasted the output from show tech... try it from the command line and see what that says.

SSH@icx6610-stack>enable
User Name:root
Password:
SSH@icx6610-stack#show stack

***** Warning! stack is not enabled. *****

T=20d16h4m43.1: alone: standalone, D: dynamic cfg, S: static
ID Type Role Mac Address Pri State Comment
1 S ICX6610-48P alone 748e.f8dc.ae80 128 local None:0


+---+
2/1| 1 |2/6
+---+
Current stack management MAC is 748e.f8dc.ae80
SSH@icx6610-stack#

as far as the ecc errors. I remember specifically seeing that in one of the crash reports - however like I said changing the power source solved the issue. @fohdeesha did point me at some memory on the bay to buy if I really felt that was the issue. still have the sticks somewhere in the basement mess...
 
Last edited: