I snagged a 'new in box' 7150-c12 for cheap. It booted clean with no errors, had all the accessories. May actually have been new in box. I booted it to Linux and am using nanddump to a USB drive to see if straight cloning the mtd partitions over to my switch with the bad certs might fix it. Since the bad switch kernel panics when any ethernet links are brought up, it's pretty much a brick with pretty lights anyway, so I have nothing to lose other than time.
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Started with a couple of PoE injectors, three Netgear switches (for general purpose, PoE and 10GbE) - now thanks to @fohdeesha & eBay, was able to consolidate everything with just a ICX6610-24. The big server in the middle connects to the rear 40 GbE port. Idle power draw is a bit higher - from 195W to 245W. But considering other folks in my workplace have 42u full size racks for 'homelabs' - this is pretty awesome.
It's sad that the ICX6 series doesn't support 802.3bt PoE standards - the ICX7 series that do use the same RPS-16 PSU ...
It's sad that the ICX6 series doesn't support 802.3bt PoE standards - the ICX7 series that do use the same RPS-16 PSU ...
I was thinking, have you tried flashing to a previous codetrain? the TPM crap is very new, there's no way it was in the 8030 train - try flashing that and see if it stops bitching? if it works then go up to 8060 then 8070 until it complains again. if the switch is freezing when links come up you'll have to flash an image from the bootloader (pretty easy)
edit: well shit, 8030 supports every 7 series except the 7150. I would hunt down the closest after that, can't remember what it is (don't remember 8040 being a thing)
Last edited:
the main reason being the ICX6xxx line was designed and released several years before 802.3bt was invented
Both 7150's I had originally had 8060 on them. I do have the images and I could have sworn that I tried a downgrade first. 8060 GA is also the oldest release listed for download from Ruckus.
Edit: the NIB unit I've imaged shipped with 8080d.
Edit: the NIB unit I've imaged shipped with 8080d.
OK. I must have used incorrect nandwrite options because it marked huge swaths of the NAND as bad, and wouldn't boot. Performed a scrub (I know, not recommended) and after that only two blocks are showing errors. Reloaded SPS8060 to primary and secondary, reloaded matching uboot and reset. Booted up, had to rollback PoE firmware (man that thing spams very quickly).
So it's up and running with switching firmware, and no TPM related messages. PoE downgrade appears to have not worked; as soon as PoE module initializes, it starts spewing the all ports lost power error. I've downloaded 8060a, 8070c, 8080b, and 8080d, and am going to step through. 8060 only has SPS image, with 8070 I'll be switching to SPR.
So it's up and running with switching firmware, and no TPM related messages. PoE downgrade appears to have not worked; as soon as PoE module initializes, it starts spewing the all ports lost power error. I've downloaded 8060a, 8070c, 8080b, and 8080d, and am going to step through. 8060 only has SPS image, with 8070 I'll be switching to SPR.
On SPR8070c. Booted clean, PoE firmware update to 2.1 went in and PoE spam stopped. No TPM errors, but 'dm verify-device-certs' throws a failure message. The 'dm create_device_profile_and_trustpoint' command is not present in 8070c. Lighting up a regular switchport does not result in a kernel panic either. On to 8080!
heeelll yeah. You should tag the other dude who had the same issue, I don't remember who it was
Could you post the commands? Might prove useful in the future as I haven't done it before. Thinking bootloader/U-boot lives in the flash, couldn't you potentially wipe them out?
from my 7250 notes, I'm pretty sure the icx7150 is identical:
Code:
(Main flash is 2GB of NAND - stores everything but bootloader)
(8MB of separate SPI flash present only for bootloader/u-boot)8080b
But it's not throwing the TPM error on startup, nor is is crashing even though I'm working with a data port connected rather than just the management port. I booted it to Linux and the /opt/tpm folder only has the system.data file in it. But it appears stable, so I updated to 8080d.
8080d (not UFI):
Same results as 8080b. No TPM error spam on boot, but same output from the commands as above. Going to see if copying over the mfg-wrapped-key.pem file solves it.
Maybe it was a weird-ass NAND issue the whole time that was fixable by scrubbing?
@klui
The nand commands I used were done in uboot and I immediately reloaded the uboot and firmware images while in uboot itself before issuing a reset.
Here's what I did (only use the scrub step as a last resort; I had a lot of pages marked as bad that were not bad):
@TheCodeLife:
Looks like nuking the NAND and starting over all the way at the bottom with SPS8060 GA and then step upgrading has fixed it for me? I don't know if your unit was as far gone as mine, or if it worked with no issues despite the TPM failure.
Code:
ICX7150-C12 Router#dm verify-device-certs
Commencing sanity check for device certs ...
Verifying TPM files ...
Failed: Check TCSD_PS Files
ICX7150-C12 Router#dm create_device_profile_and_trustpoint
PKI: Error in opening certificate file - Manufacturing certificate file.
Error: File not found
Info: Device certificate import is failed ..!!, ret: 16
Error: read_private_key_from_tpm, Private key file ../opt/tpm/mfg-wrapped-key.pem does not exists...!!
pki_import_device_key_file, load tpm private key is failed..!!
Error: key do not exist
Info: Device lable creation is failed ..!!, ret :248080d (not UFI):
Same results as 8080b. No TPM error spam on boot, but same output from the commands as above. Going to see if copying over the mfg-wrapped-key.pem file solves it.
Maybe it was a weird-ass NAND issue the whole time that was fixable by scrubbing?
@klui
The nand commands I used were done in uboot and I immediately reloaded the uboot and firmware images while in uboot itself before issuing a reset.
Here's what I did (only use the scrub step as a last resort; I had a lot of pages marked as bad that were not bad):
Code:
nand erase.chip clean
nand scrub.chip (only if you are certain that there are pages incorrectly marked as bad)
nand erase.chip clean (only if just scrubbed again just to get it to try to write each page after the scrub so it would update the bad block table)
setenv ipaddress 192.168.0.209 (address of switch)
setenv serverip 192.168.0.14 (address of TFTP server)
setenv uboot mnz10114.bin (filename of uboot image)
setenv image_name SPR8080d.bin (filename of firmware image to load)
update_uboot
update_primary
update_secondaryLooks like nuking the NAND and starting over all the way at the bottom with SPS8060 GA and then step upgrading has fixed it for me? I don't know if your unit was as far gone as mine, or if it worked with no issues despite the TPM failure.
That's a relief. I haven't looked at your notes about other switches and I appreciate the reminder.
Damn:
Buuuut it's different from the previous failure. Last time I was getting this, it did not include "PKI: File not loaded, Manufacturing certificate expired." in the data.
So I configured NTP, got it to have the correct time and that resulted in this:
Welp.
Booted to Linux, delete the PEM files, restarted, reran the command and it predictably failed but did not kernel panic the switch. I issued a factory reset, as I had disabled some logging to console earlier. About every 15 seconds I get the message "Info: Device Key or Cert file is not available" even though I issued "no logging buffered informational" and "no logging buffered notifications"
But despite that and still having a data port connected by which I've been transferring data, it hasn't crashed under normal use. I need to throw some extra test boxes behind it and saturate a couple of ports to see what happens.
Code:
ICX7150-C12 Router#dm verify-device-certs
Commencing sanity check for device certs ...
Verifying TPM files ...
Failed: Check TCSD_PS Files
ICX7150-C12 Router#dm create_device_profile_and_trustpoint 2788017248:error:8006B06D:tpm engine:TPM_LOAD_SRK:request failed:e_tpm.c:278:
2788017248:error:8006F07A:tpm engine:TPM_ENGINE_LOAD_KEY:failed loading the SRK:e_tpm.c:637:
2788017248:error:26096080:lib(38):func(150):reason(128):NA:0:
/vobs/fdry/build/../../../../..///vobs/mucho/mp/cmds/web_cmds.c:1605 Couldn't load TPM key "../opt/tpm/mfg-wrapped-key.pem" from file.
PKI: File not loaded, Manufacturing certificate expired.
Error: Certificate is invalid
Info: Device certificate import is failed ..!!, ret: 21
pki_import_device_key_file, load tpm private key is failed..!!
Error: key do not exist
Info: Device lable creation is failed ..!!, ret :24So I configured NTP, got it to have the correct time and that resulted in this:
Code:
ICX7150-C12 Router#dm create_device_profile_and_trustpoint
stack: 0147b7e4 01456b14 b6b937c1 33ff2fe0
[ 458.360012] [BrcdSoftlockup]: sim_softwatchdog thread is detached on core=0
Application received signal -> SIGNUM#11
Tuning CFS scheduler parameters...
Copying fitrace errorlog file to flash
CORE_PATTERN:PID=1046 UID=0 GID=0 sig=11
Thu Jul 16 23:44:34 UTC 2020: Dumping core file to /tmp.gz, this will take couple of minutes ...Booted to Linux, delete the PEM files, restarted, reran the command and it predictably failed but did not kernel panic the switch. I issued a factory reset, as I had disabled some logging to console earlier. About every 15 seconds I get the message "Info: Device Key or Cert file is not available" even though I issued "no logging buffered informational" and "no logging buffered notifications"
But despite that and still having a data port connected by which I've been transferring data, it hasn't crashed under normal use. I need to throw some extra test boxes behind it and saturate a couple of ports to see what happens.
I wonder if your primary is failing. I have some old Nortel 5510/5520 switches and one of them had issues saving their config or a similar problem. I started using the secondary only and for subsequent deployments for production don't log too much into their NAND, and send events to a remote server.
I'm curious, in the 7X50 lineup, how was each series positioned relative to the other as far as market segment goes?
7150 - fixed 24/48 1G copper port configurations; maximum of 8x 10 GB ports (4x SFP/SFP+ on most, 8 on 48ZP, 2 on C12); single fixed power supply; fixed fans; ZP models are multigigabit
7250 - fixed 24/48 1G copper port configurations; maximum of 8 10 GB ports (8x SFP/SFP+); single fixed power supply; fixed fans
7450 - fixed 24/48 1G copper port configurations but some multigig and PoH options; slots for 4x10G or 1x 40G modules; HS redundant PSU and fan modules; ZP models are multigigabit
7650 - I haven't really looked at these so can't say; very few of them on eBay
7750 - 48 ports copper or SFP+ 10 G + 6x 40G ports; optional second 6x 40G module in rear; HS redundant PSU and fan modules; can act as campus fabric controller (other 7x50 switches can be added as port extenders instead of stand alone switches)
7850 - appears to have multiple 48 port configurations as either SFP+ or 10/25 G SFP28 + 8x 40/100G QSFP28 ports; I would assume same power options as 7750.
7750 and 7850 have no PoE options; all the other switches do.
www.ruckussecurity.com
7250 - fixed 24/48 1G copper port configurations; maximum of 8 10 GB ports (8x SFP/SFP+); single fixed power supply; fixed fans
7450 - fixed 24/48 1G copper port configurations but some multigig and PoH options; slots for 4x10G or 1x 40G modules; HS redundant PSU and fan modules; ZP models are multigigabit
7650 - I haven't really looked at these so can't say; very few of them on eBay
7750 - 48 ports copper or SFP+ 10 G + 6x 40G ports; optional second 6x 40G module in rear; HS redundant PSU and fan modules; can act as campus fabric controller (other 7x50 switches can be added as port extenders instead of stand alone switches)
7850 - appears to have multiple 48 port configurations as either SFP+ or 10/25 G SFP28 + 8x 40/100G QSFP28 ports; I would assume same power options as 7750.
7750 and 7850 have no PoE options; all the other switches do.
RUCKUS ICX Family Switches | NetWifiStore.com
www.ruckussecurity.com
Last edited:
The ZP models are quite new - they support 2.5G / 5G / 10G and they support 802.3bt
Initially looked at this thread and the 7150 compact models and was confused when I googled them and the ZP series came up with 2.5G/5G support
Yeah the c10-ZP kinda sad that the two SFP uplinks are gig only. Would have been nice to have 2 10G capable PoH ports and also have 2 uplinks worth using. Or 2 QSFP uplinks. I've seen switches in that size range that have QSFP. Guess they didn't want it to compete with the 48ZP, but the 48ZP has no PoH ports that do anything more than 2.5 Gbps.The ZP models are quite new - they support 2.5G / 5G / 10G and they support 802.3bt
Initially looked at this thread and the 7150 compact models and was confused when I googled them and the ZP series came up with 2.5G/5G support
I think the 7650 and 7850 are later additions, they aren't mentioned in a lot of the literature I've found.