$127 Cisco ENCS5412/K9 Xeon-D 1557 (12 core), 32G ram

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

iGene

Member
Jun 15, 2014
74
23
8
Taiwan
There's an NIM expansion slot on it, but I couldn't find any information on if C-NIM-1X will work.

If it works you can get 10G on it which will make it a better deal.
 

ccie4526

Active Member
Jan 25, 2021
139
115
43
I have been playing with one of these for a few weeks, I was actually working on documenting some of my findings but haven't gotten far enough to post something yet :p but I can answer some questions

It runs ESXi 8
Curious, you are running this instead of the NFVIS firmware? Standard install from USB or ??
 

RobstarUSA

Active Member
Sep 15, 2016
235
104
43
There's an NIM expansion slot on it, but I couldn't find any information on if C-NIM-1X will work.

If it works you can get 10G on it which will make it a better deal.
If anyone tries this, please post results :)
 

turbo

Member
Mar 17, 2022
30
34
18
So they have a SoC with decent ports and they just ignore them? That's just rude. Bloody CISCO.
Yep I thought the same thing. I'm guessing the XL710 does something(s) that the X552 does not, they do have a custom firmware for it.

It's a bunch of I2C/MDIO commands to tell it how to configure itself, isn't it? I have some hopes more switch chips get upstream support with Switchdev but so far it's a bit of a shitshow and almost all non-mellanox chips need some device tree hints for automatic detection.

It'd be interesting to see what I2C, SPI and MDIO (or other sideband) interfaces the thing has. PoE, port personality and perhaps hardware control like power, fans and LEDs could live on some magic FPGA I2C address.
It seems to be something inside their modified I350 driver but I haven't really looked at exactly what it's doing yet. I did try to switch it with ethtool and cli in ESXi (localcli network nic set -p tp -n vmnic0) but neither did anything. I did submit an opensource request hoping to get some of their modifications, but no joy yet.
 

turbo

Member
Mar 17, 2022
30
34
18
Curious, you are running this instead of the NFVIS firmware? Standard install from USB or ??
Yep, I don't have access to a recent NFVIS image but mine came with an image of NFVIS 3.8 or so. "Fragile" is the best word I could use to describe my NFVIS experience but ESXi has been solid. I am running from a USB stick right now, so I can pass the SATA controller through to TrueNAS but I'll probably move ESXi to the internal 200GB SATA drive then use RDM for a couple SATA SSDs I have in there.
 

oneplane

Well-Known Member
Jul 23, 2021
872
529
93
The NFVIS will probably contain the data needed to construct I2C/MDIO commands to manage PoE, port personality and LEDs. Should be possible to extract, it'll be in init scripts, systemd units, or some custom binaries that are easily reversed to get the register addresses.
 
  • Like
Reactions: Aluminat

Mymlan

Clean, Friendly, and In Stock.
Oct 1, 2013
35
81
18
Has anyone tried a fan swap? Will the CIMC get aggressive or is it just a hardware warning?
 

turbo

Member
Mar 17, 2022
30
34
18
The NFVIS will probably contain the data needed to construct I2C/MDIO commands to manage PoE, port personality and LEDs. Should be possible to extract, it'll be in init scripts, systemd units, or some custom binaries that are easily reversed to get the register addresses.
The switch module is actually configured through IP (specifically 169.254.1.0)

There is a dedicated VLAN 2363 used by the host to communicate with the switch for configuration. It is a full managed switch with LACP, VLANs, POE, QinQ, QoS, etc lots of features, all controlled through a terribly inefficient (but easy to understand) XML interface.

Switch is bootstrapped by writing the bootloader and software image through an MMIO range on the host and sending an interrupt. This is handled by a special driver "mv_pciboot" and an app called "remote_boot_app" I do this on my unit by using PCIPassThru on ESXi to a VM that performs this process once per HW power-on. I haven't found a way to reset the switch once it's up and running yet (for instance if you botch the configuration and lose access to it via IP)

As for the LEDs, port personalities, and other HW settings I haven't gotten that far yet.
 

oneplane

Well-Known Member
Jul 23, 2021
872
529
93
The switch module is actually configured through IP (specifically 169.254.1.0)

There is a dedicated VLAN 2363 used by the host to communicate with the switch for configuration. It is a full managed switch with LACP, VLANs, POE, QinQ, QoS, etc lots of features, all controlled through a terribly inefficient (but easy to understand) XML interface.

Switch is bootstrapped by writing the bootloader and software image through an MMIO range on the host and sending an interrupt. This is handled by a special driver "mv_pciboot" and an app called "remote_boot_app" I do this on my unit by using PCIPassThru on ESXi to a VM that performs this process once per HW power-on. I haven't found a way to reset the switch once it's up and running yet (for instance if you botch the configuration and lose access to it via IP)

As for the LEDs, port personalities, and other HW settings I haven't gotten that far yet.
Nice! I guess Cisco didn't want to bother with the sideband interface and went straight for in-band. This probably also means the payload on the Cisco OS contains all the options/strings/operations that would be exposed once the configuration is loaded into the switch.

What's strange about that switch chip is that the spreadsheet does mention a programmable core, but nothing about it being a fully managed switch that operates stand-alone.
 

devz3ro

New Member
Nov 16, 2019
22
2
3
Bought a few of these for a lab setup. If anyone could supply the latest firmware / NFVIS etc. I would appreciate it, please PM if you have access to it. Thanks in advance.
 

TLN

Active Member
Feb 26, 2016
523
84
28
35
I have a bunch of these, and if you live in the SF Bay Area and can pick up, I can give you a pretty good deal on them if you want one. I bought six of them from the same seller for a cluster that I played with but ultimately ended up going in a different direction.

Having said that -- these are a bit of an oddball. You can install whatever OS you want on them, but there are some features (like PoE on the left-hand side switch) that I believe you can only enable using the Cisco software, which you need a support contract for. I can provide some pictures of the inside if you are curious -- perhaps someone who is a bit more experienced with Cisco stuff can help me figure out how to get PoE working.

Idle power usage is about 60 watts. There's no way to use a PCIe or OCP card as far as I can tell, so you are stuck with 1Gb/s.
PM coming
 

Jorge Perez

Active Member
Dec 8, 2019
113
51
28
Almost bought one of these, then i realized it was a doorstop and managed to cancel.

Don't understand why it would have 1G instead of 10G.
 

ccie4526

Active Member
Jan 25, 2021
139
115
43
How do you update the bios on these? Seems my bios has a password on it.
Some default passwords for the ENCS 5400 are "password" and "Admin123#", sans quotes of course. The default worked on mine.

Almost bought one of these, then i realized it was a doorstop and managed to cancel.

Don't understand why it would have 1G instead of 10G.
The ENCS5400s are older units and were replaced with the Catalyst 8200s. I was about to say that the 8200s support 10G, but I figured I'd better check the data sheet before I said something wrong, and sure enough, the 8200s only support 1G connections as well, but DOES support a 2.5Gbps NIM. At the edge, one generally doesn't need much more than 2.5Gbps anyway.... or if you do, you're using a dedicated platform, not one of these things.
 
  • Like
Reactions: blakwolf