I don't even think jtag would fix that, it seems the bootloader is intact but something has gone wrong enough with the CPU, RAM, or something else that's it's going into reset as soon as it tries to run anything. you could try reseating the RAM as a last hopeI recently acquired another ICX6610-48P from eBay for dirt cheap. It was sold as being bricked ("It stopped booting fully after a firmware upgrade"), but that's fine with me - my primary reason for purchasing it was to swap the 40 GbE daughtercard into my existing, functional 6610 that has one defective 40 GbE port.
Just for grins, though (and before I go through all the effort of taking both of them apart), I pulled a PSU and fan tray from my working unit and stuck it in this one to see what state it's in.
It's stuck in a boot loop of some kind. The PSU fans rev up upon having AC applied, run for a couple seconds, rev down for a few seconds more, and then rev back up again. Repeat x inifinity.
On the console, this is what I get (over and over again). Hitting 'a' or 'b' as prompted looks like they work, but they do nothing to interrupt the boot looping - it just reboots again a few seconds later.
Any hope of recovering this short of JTAG?Code:ICX Boot Code Version 10.1.00 (grz10100) Enter 'a' to stop at memory test Enter 'b' to stop at boot monitor Boot(Mem-Test)>aaaaaaaaaaaaaaa$ ICX Boot Code Version 10.1.00 (grz10100) Enter 'a' to stop at memory test Enter 'b' to stop at boot monitor Boot(Mem-Test)>bbbbbbb$ ICX Boot Code Version 10.1.00 (grz10100) Enter 'a' to stop at memory test Enter 'b' to stop at boot monitor ***** Interrupted by entering 'b' ***** .$ ICX Boot Code Version 10.1.00 (grz10100) Enter 'a' to stop at memory test Enter 'b' to stop at boot monitor ***** Interrupted by entering 'b' ***** .$ ICX Boot Code Version 10.1.00 (grz10100) Enter 'a' to stop at memory test Enter 'b' to stop at boot monitor
Alrighty, I guess I'm popping off the lid and having a look inside...I don't even think jtag would fix that, it seems the bootloader is intact but something has gone wrong enough with the CPU, RAM, or something else that's it's going into reset as soon as it tries to run anything. you could try reseating the RAM as a last hope
Looks like it was actually a bad PSU I was using. I had two Rev. A PSUs sitting in my shelf and I knew one of them was a dud. I assumed it was the one with the bent handle (you know, the physically damaged one...) and tried the other one. On a whim today during my lunch break I tried the other one, and it started booting!I don't even think jtag would fix that, it seems the bootloader is intact but something has gone wrong enough with the CPU, RAM, or something else that's it's going into reset as soon as it tries to run anything. you could try reseating the RAM as a last hope
$
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
BOOT INFO: load monitor from boot flash, cksum = 71f1
BOOT INFO: verify flash files..........
BOOT INFO: load image from primary copy...
platform type = 12
PCIE-1 LTSSM status: 22
PCIE Switch status: 0
.............................
Firmware integrity checksum passed
.......
Starting Main Task ...CPSS DxCh Version: cpss3.4p1 release
Pre Parsing Config Data ...
Is it unpacking with the same settings (all cores/threads vs single thread)?The FCX has the same CPU and has to unpack the exact same firmware to memory and it boots like 5x faster
the management plane between the two is identical (that's why they run the same firmware and even the same bootloader image). Both are single core single thread PPC procs booting from CFI flash, the flash part number is even the same between them (S29GL512P). the main difference is the ASIC of course, perhaps the newer marvell in the 6610 takes longer to initializeIs it unpacking with the same settings (all cores/threads vs single thread)?
Is the boot device connected in the same way?
Is the boot device on pair?
Aug 19 18:45:51:I:System: Interface ethernet 2/2/4, state down
Aug 19 18:45:51:I:STP: VLAN 1 Port 2/2/4 STP State -> DISABLED (PortDown)
Aug 19 18:45:51:I:STP: VLAN 1 Port 2/2/4 STP State -> LISTENING (PortDown)
Aug 19 18:45:50:I:System: Interface ethernet 2/2/4, state up
Aug 19 18:45:50:I:STP: VLAN 1 Port 2/2/4 STP State -> LISTENING (MakeFwding)
Aug 19 18:44:31:I:System: Interface ethernet 2/2/4, state down
Aug 19 18:44:31:I:STP: VLAN 1 Port 2/2/4 STP State -> DISABLED (PortDown)
Aug 19 18:44:31:I:STP: VLAN 1 Port 2/2/4 STP State -> LISTENING (PortDown)
Aug 19 18:44:30:I:System: Interface ethernet 2/2/4, state up
Aug 19 18:44:30:I:STP: VLAN 1 Port 2/2/4 STP State -> LISTENING (MakeFwding)
Aug 19 18:51:38:I:System: Interface ethernet 2/2/4, state down
Aug 19 18:51:37:I:System: Interface ethernet 2/2/4, state up
Aug 19 18:51:22:I:System: Interface ethernet 2/2/4, state down
Aug 19 18:51:22:I:System: Interface ethernet 2/2/4, state up
Posting your 6610 configuration would be helpful.I would be very happy if you suggest anything about the issue.
Thanks for your reply itronin.Posting your 6610 configuration would be helpful.
SSH@ICX6610-24 Switch#sh run
Current configuration:
!
ver 08.0.30tT7f1
!
stack unit 1
module 1 icx6610-24-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
priority 128
stack-port 1/2/1 1/2/6
stack unit 2
module 1 icx6610-24-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-port 2/2/1 2/2/6
stack enable
stack mac xxxx.xxxx.xx74
!
!
!
lag xxxx dynamic id 1
ports ethernet 1/3/8 ethernet 2/3/8
primary-port 1/3/8
deploy
port-name xxxx ethernet 1/3/8
port-name xxxx ethernet 2/3/8
!
lag xxxx dynamic id 2
ports ethernet 1/3/3 ethernet 2/3/3
primary-port 1/3/3
deploy
!
lag xxxx dynamic id 3
ports ethernet 1/3/4 ethernet 2/3/4
primary-port 1/3/4
deploy
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan xxxx name xxxx by port
tagged ethe 1/3/1 ethe 1/3/8 ethe 2/3/1 ethe 2/3/8
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/3/1 ethe 1/3/8 ethe 2/3/1 ethe 2/3/8
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
spanning-tree 802-1w
!
vlan xxxx name xxxx by port
tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
spanning-tree 802-1w
!
!
!
!
!
aaa authentication enable default local
aaa authentication login default local
boot sys fl sec
jumbo
enable telnet authentication
ip address x.x.x.x x.x.x.x
ip dns server-address 8.8.8.8
no ip dhcp-client enable
ip default-gateway x.x.x.x
!
username xxxx password .....
!
!
clock timezone gmt GMT+03
!
!
ntp
disable serve
server 192.36.143.130
server 213.251.52.185
!
!
hitless-failover enable
interface ethernet 1/1/17
dual-mode xxxx
!
interface ethernet 1/2/1
port-name xxxx
!
interface ethernet 1/3/1
port-name xxxx
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
dual-mode xxxx
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
port-name xxxx
speed-duplex 10G-full
!
interface ethernet 2/2/4
no spanning-tree
!
interface ethernet 2/3/1
speed-duplex 10G-full
!
interface ethernet 2/3/2
speed-duplex 10G-full
!
interface ethernet 2/3/5
speed-duplex 10G-full
!
interface ethernet 2/3/6
speed-duplex 10G-full
!
interface ethernet 2/3/7
speed-duplex 10G-full
!
!
!
!
!
!
ip ssh timeout 60
!
!
end
Fan 1 ok, speed (auto): 1<->[[2]]
Fan 2 ok, speed (auto): 1<->[[2]]
Fan 3 ok, speed (auto): 1<->[[2]]
Fan controlled temperature:
Rule 1/2 (MGMT THERMAL PLANE): 51.7 deg-C
Rule 2/2 (AIR OUTLET NEAR PSU): 34.0 deg-C
Fan speed switching temperature thresholds:
Rule 1/2 (MGMT THERMAL PLANE):
Speed 1: NM<-----> 95 deg-C
Speed 2: 85<----->105 deg-C (shutdown)
Rule 2/2 (AIR OUTLET NEAR PSU)***active***:
Speed 1: NM<-----> 41 deg-C
Speed 2: 34<----->105 deg-C (shutdown)
Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back
Fan 3 Air Flow Direction: Front to Back
Slot 1 Current Temperature: 51.7 deg-C (Sensor 1), 34.0 deg-C (Sensor 2)
Slot 2 Current Temperature: NA
Warning level.......: 100.0 deg-C
Shutdown level......: 105.0 deg-C
I just finished doing the exact same modification this afternoon on my 7250-48P but instead of the Sunon fan I replaced it with an EFB0612LA-R00 because Digi-Key was out of stock on the Sunon.I have implemented the @RoachedCoach Mechatronics and Sunon fan mods discussed here on a ICX7250-48P. The new fans are definitely quieter, but I am not getting the results that others have. From a cold boot, the ASIC temps rise to mid-60's which is no problem, however the air outlet temps rise to the speed 2 threshold, the fans go to high and the temp drops to exactly 34c, the threshold for dropping back to speed 1. So once speed 2 becomes active, it never drops back. I tried just running with 1 fan, air outlet temps go into the 60's, but still have the one fan running at high speed. With the fan(s) running at high speed, the 7250-48P is louder than a 6610-24F with 2 Rev. A power supplies.
Due to a comedy of seller errors, I also have a 7250-24 and 7250-24P with dead PoE. The 7250-24 with its single OEM fan runs happily in the same rack with the fan at low speed. But I am not fond of the idea of going back to PoE power bricks.
Any ideas on getting a 7250-48P cool enough so the fans run at low speed?
ICX6450-48P Switch#show chassis
The stack unit 1 chassis info:
Power supply 1 (NA - AC - Regular) present, status ok
Power supply 2 not present
Power supply 3 not present
Fan 1 failed
Fan 2 failed
Fan 3 failed
Fan controlled temperature: 41.5 deg-C
Fan speed switching temperature thresholds:
Speed 1: NM<----->61 deg-C
Speed 2: 56<-----> 79 deg-C (shutdown)
Sensor B Temperature Readings:
Current temperature : 37.0 deg-C
Sensor A Temperature Readings:
Current temperature : 41.5 deg-C
Warning level.......: 69.0 deg-C
Shutdown level......: 79.0 deg-C
it's probably showing failed because the fans are spinning slower than the "failed" threshold RPM. to check if it's still getting RPM data just run "dm fan-speed". Either way on the 6450 it doesn't matter, as long as your temps are good. if it were a 7 series it'd be refusing to boot because of it but the 6 series doesn't give a shiza about no dang ole fans
All fans in Low speed (0x83)
Looks like you have all of your VLANs tagged on just about every port on these switches. Is that what you mean to do?Thanks for your reply itronin.
Code:SSH@ICX6610-24 Switch#sh run Current configuration: ! ver 08.0.30tT7f1 ! stack unit 1 module 1 icx6610-24-port-management-module module 2 icx6610-qsfp-10-port-160g-module module 3 icx6610-8-port-10g-dual-mode-module priority 128 stack-port 1/2/1 1/2/6 stack unit 2 module 1 icx6610-24-port-management-module module 2 icx6610-qsfp-10-port-160g-module module 3 icx6610-8-port-10g-dual-mode-module stack-port 2/2/1 2/2/6 stack enable stack mac xxxx.xxxx.xx74 ! ! ! lag xxxx dynamic id 1 ports ethernet 1/3/8 ethernet 2/3/8 primary-port 1/3/8 deploy port-name xxxx ethernet 1/3/8 port-name xxxx ethernet 2/3/8 ! lag xxxx dynamic id 2 ports ethernet 1/3/3 ethernet 2/3/3 primary-port 1/3/3 deploy ! lag xxxx dynamic id 3 ports ethernet 1/3/4 ethernet 2/3/4 primary-port 1/3/4 deploy ! ! vlan 1 name DEFAULT-VLAN by port ! vlan xxxx name xxxx by port tagged ethe 1/3/1 ethe 1/3/8 ethe 2/3/1 ethe 2/3/8 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/3/1 ethe 1/3/8 ethe 2/3/1 ethe 2/3/8 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7 spanning-tree 802-1w ! vlan xxxx name xxxx by port tagged ethe 1/1/1 to 1/1/24 ethe 1/2/2 to 1/2/5 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/24 ethe 2/2/2 to 2/2/3 ethe 2/2/5 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7 spanning-tree 802-1w ! ! ! ! ! aaa authentication enable default local aaa authentication login default local boot sys fl sec jumbo enable telnet authentication ip address x.x.x.x x.x.x.x ip dns server-address 8.8.8.8 no ip dhcp-client enable ip default-gateway x.x.x.x ! username xxxx password ..... ! ! clock timezone gmt GMT+03 ! ! ntp disable serve server 192.36.143.130 server 213.251.52.185 ! ! hitless-failover enable interface ethernet 1/1/17 dual-mode xxxx ! interface ethernet 1/2/1 port-name xxxx ! interface ethernet 1/3/1 port-name xxxx speed-duplex 10G-full ! interface ethernet 1/3/2 speed-duplex 10G-full ! interface ethernet 1/3/3 speed-duplex 10G-full ! interface ethernet 1/3/4 speed-duplex 10G-full ! interface ethernet 1/3/5 dual-mode xxxx speed-duplex 10G-full ! interface ethernet 1/3/6 speed-duplex 10G-full ! interface ethernet 1/3/7 speed-duplex 10G-full ! interface ethernet 1/3/8 port-name xxxx speed-duplex 10G-full ! interface ethernet 2/2/4 no spanning-tree ! interface ethernet 2/3/1 speed-duplex 10G-full ! interface ethernet 2/3/2 speed-duplex 10G-full ! interface ethernet 2/3/5 speed-duplex 10G-full ! interface ethernet 2/3/6 speed-duplex 10G-full ! interface ethernet 2/3/7 speed-duplex 10G-full ! ! ! ! ! ! ip ssh timeout 60 ! ! end
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 10 name Secure by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/3/1 to 1/3/2
untagged ethe 1/1/3 to 1/1/5 ethe 1/1/7 ethe 1/1/9 to 1/1/11 ethe 1/1/13 to 1/1/36 ethe 1/2/2 to 1/2/5 ethe 1/3/3 to 1/3/5
router-interface ve 10
!
vlan 20 name Gizmos by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/3/1 to 1/3/2
untagged ethe 1/1/37 to 1/1/38
router-interface ve 20
!
vlan 30 name WiFi by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/3/1 to 1/3/2
untagged ethe 1/1/45 to 1/1/48
router-interface ve 30
!
vlan 40 name Cameras by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/3/1 to 1/3/2
untagged ethe 1/1/39 to 1/1/44
router-interface ve 40
!
vlan 1000 name Storage by port
untagged ethe 1/1/6 ethe 1/1/8 ethe 1/1/12 ethe 1/2/7 to 1/2/10 ethe 1/3/6 to 1/3/8
router-interface ve 1000
!
spec sheet for the OEM psu model number I have for the 6450 says 100vac - 130vac but it's probably a bit more lenient than that. That's really bad voltage drop though, is that circuit in your house really loaded down? Even with poor wiring you shouldn't be dropping nearly 30 volts between the breaker panel and the outlet unless you're doing like 20 amps over it. Can you measure the incoming power at your breaker panel or meter? If it's that low there that's usually bad enough that the power company will do something about it. you could be at the end of a very long run after the nearest stepdown transformer or somethingThis is with only 2/3 fans connected to the 6450-48p btw. Will see how temps look once it actually has work to do.Code:All fans in Low speed (0x83)
BTW, my power meter between the switch and outlet reports only 97-99V, any idea how sensitive it is for undervoltage? Old house with likely shitty wiring.
That's a good recommendation, will try that!spec sheet for the OEM psu model number I have for the 6450 says 100vac - 130vac but it's probably a bit more lenient than that. That's really bad voltage drop though, is that circuit in your house really loaded down? Even with poor wiring you shouldn't be dropping nearly 30 volts between the breaker panel and the outlet unless you're doing like 20 amps over it. Can you measure the incoming power at your breaker panel or meter? If it's that low there that's usually bad enough that the power company will do something about it. you could be at the end of a very long run after the nearest stepdown transformer or something
That's a good recommendation, will try that!
I'm in Japan though, where 100V is standard, so the drop is not *that* drastic.