Yeah I messed around with it yesterday and it doesn't seem to show up on the 8000v I can't find anything showing a list of what modules actually work with the 8000v but the NIM-ES2-8 isn't listed as compatible with the ENCS (running NFVIS that is) so it might just not work. I've done the VM based solution that's listed earlier in this thread but it's less than ideal with how complicated it is to code for the port configurations. I'm not sure if there's some place that the NFVIS actually stores the switch config XML maybe I can generate the file on a regular NFVIS and then export it to my switch bootstrap VM or something but honestly even if the switch module is useless I doubt I'll ever need it for anything. I'm also kind of curious at how the USB port presents itself to the OS and maybe I can get a 2.5G USB 3.0 ethernet adapter to work on it but generally their performance isn't that great anyway. You can find the NIM-1GE-CU-SFP on ebay for ~$20 so I might just pick up one of those the 2 port modules still fetch a much higher price at ~$125. Mine came with a 4 T1/E1 module in the slot but that's pretty much useless to me.
$127 Cisco ENCS5412/K9 Xeon-D 1557 (12 core), 32G ram
- Thread starter RobstarUSA
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
You can actually setup NFVIS inside a VM, pass through a XL710 virtual instance configured for VLAN 4095 (vmware magic code for pass-through-all-VLANs) and use it to configure the switch.
I had planned to create a VM with a cli to configure the switch module, but once I configured it for 2 ports LACP and 6 ports 802.1q trunks I have not needed to change it in a year
The fact that just bootstrapping the switch module enables the XL710 VM-VM acceleration is sweet.
NFVIS does keep the configuration in some kind of database and they have a bunch of python code that pulls that to create the XML to configure the switch. It's ugly.
I had planned to create a VM with a cli to configure the switch module, but once I configured it for 2 ports LACP and 6 ports 802.1q trunks I have not needed to change it in a year
The fact that just bootstrapping the switch module enables the XL710 VM-VM acceleration is sweet.NFVIS does keep the configuration in some kind of database and they have a bunch of python code that pulls that to create the XML to configure the switch. It's ugly.
There was 1 thing I changed, in the .vmx file under ESXi I set
before installing NFVIS. Other than that, I just added an SRIOV passthrough for PCI device 00000:008:00.1
NFVIS has drivers for the VF which enables it to talk to the Marvel switch through the XL710 via IP.
Code:
SMBIOS.reflectHost = "TRUE"NFVIS has drivers for the VF which enables it to talk to the Marvel switch through the XL710 via IP.
Has anyone been able to extract the latest BIOS and CIMC firmware? 4.16.1 upgrades the Bios and CIMC software when you install it via the ISO. But digging around I can't seem to find the files anywhere in the ISO image. I'm not sure what the latest BIOS is but the latest CIMC version is like 4.0(something).
Been playing with a NIM module and it seems like whatever interface cisco uses it does not present to the OS. Or at least I can't seem to find it in Proxmox. The network interface doesn't detect nor does it show up in the list of PCI devices that I can pass through to the VM.
For anyone interested... found them in "\Packages\nfvis-fwupdate-4.16.1-FC2.el8.el8.x86_64.rpm" open it up in some sort of zip program and navigate to opt\fwupdate\ and you'll find two files "CIMC_3.2.14.19.bin" and ENCS54_BIOS_4.00.SPA
That seems to be expected even if you bootstrap the switch - which is what I was hoping would reveal the NIM so I could add an LTE card in for built-in failover.
I have two of these still that I am debating using for a "low power lab" only because I am kneecapped by some of the hardware/software flexibility.
Ok, so I finally made some progress in getting an NFVIS VM running on Proxmox. I created a VM, I had to choose 2.4 Linux kernel, Make the hard drive 32G SATA, Add a serial port, passed through the Marvell 0e:00 PCIe device and the 08:00 x710 interfaces.
Had to use the xterm.js console and the serial port to do some troubleshooting, but finally got it to go through the install disk for 4.16.1 but... I'm having issues getting the interfaces to come up. They get assigned eth0, eth1 etc instead of GE0-0, GE0-1 etc.Iwas able to change the IP and what interface is bound to lan-br and I got the PNIC up for eth0 by using an E1000E interface so I can actually web into the box now but it doesn't seem to recognize the switch module and the x710 interfaces are down and even tried admin-downing them and then bringing them back up with no luck.
It regognizes the system as a C8200-UCPEVM I tried adding the SMBIOS info from the host to the VM but it doesn't seem to see that.
I also do not see the switch module in NFVIS at all (not even in the menus) and I still can't find the NIM module to see if that can possibly passthrough.
I tried setting the SMBIOS type 1 to what I got from the ENCS itself and using qm set <VM Number> --args "-smbios type=0,version= ENCS54_4.00.040420241525" to get the bios version set for an ENCS but it didn't work it still knows it's a VM and thinks it's running on an 8200.
Had to use the xterm.js console and the serial port to do some troubleshooting, but finally got it to go through the install disk for 4.16.1 but... I'm having issues getting the interfaces to come up. They get assigned eth0, eth1 etc instead of GE0-0, GE0-1 etc.Iwas able to change the IP and what interface is bound to lan-br and I got the PNIC up for eth0 by using an E1000E interface so I can actually web into the box now but it doesn't seem to recognize the switch module and the x710 interfaces are down and even tried admin-downing them and then bringing them back up with no luck.
It regognizes the system as a C8200-UCPEVM I tried adding the SMBIOS info from the host to the VM but it doesn't seem to see that.
I also do not see the switch module in NFVIS at all (not even in the menus) and I still can't find the NIM module to see if that can possibly passthrough.
I tried setting the SMBIOS type 1 to what I got from the ENCS itself and using qm set <VM Number> --args "-smbios type=0,version= ENCS54_4.00.040420241525" to get the bios version set for an ENCS but it didn't work it still knows it's a VM and thinks it's running on an 8200.
On my ESXi NFVIS VM I have 4 E1000e interfaces plus the passed through X710 VFs. I bootstrap the switch with a different VM before NFVIS starts because I have other VMs using the internal switch and I didn't want to depend on NFVIS to do it.
The interface mapping happens in /etc/udev/rules.d/70-persistent-net.rules, I think I did change this to map to the MAC address instead of using the PCI ID because that changed for some reason when I was playing with different options. something like this
the real SSH port is 22222 at least on my NFVIS version. You have to clear the fw rules before it will respond though (iptables --flush works)
dmidecode reports this inside the NFVIS VM in case this helps to replicate it
The interface mapping happens in /etc/udev/rules.d/70-persistent-net.rules, I think I did change this to map to the MAC address instead of using the PCI ID because that changed for some reason when I was playing with different options. something like this
ACTION=="add", SUBSYSTEM=="net", ATTR{address}=="00:0c:29:xx:xx:xx", NAME="MGMT"the real SSH port is 22222 at least on my NFVIS version. You have to clear the fw rules before it will respond though (iptables --flush works)
dmidecode reports this inside the NFVIS VM in case this helps to replicate it
Code:
Handle 0x0000, DMI type 0, 26 bytes
BIOS Information
Vendor: VMware, Inc.
Version: VMW201.00V.20829224.B64.2211211842
Release Date: 11/21/2022
ROM Size: 2048 kB
Characteristics:
ISA is supported
PCI is supported
PNP is supported
BIOS is upgradeable
ACPI is supported
Targeted content distribution is supported
UEFI is supported
Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: Cisco Systems, Inc.
Product Name: ENCS5412/K9
Version: None
Serial Number: FGLxxxxx07K
UUID: xxxxxxxxx-xxxx-xxxx-xxxx-881e6fda766d
Wake-up Type: Power Switch
SKU Number: Not Specified
Family: Not Specified
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: Intel Corporation
Product Name: 440BX Desktop Reference Platform
Version: None
Serial Number: None
Asset Tag: Not Specified
Features: None
Location In Chassis: Not Specified
Chassis Handle: 0x0000
Type: Other
Contained Object Handles: 0
Handle 0x0003, DMI type 3, 21 bytes
Chassis Information
Manufacturer: No Enclosure
Type: OtherBeen a long time since I've been active on sth.
On a lark, I stumbled across an insanely good deal on the replacement for these devices – the C8200-UCPE.
Gonna see if I can't get esxi going on that device and will report back. But if someone's done it already, would love to hear if there were any issues.
On a lark, I stumbled across an insanely good deal on the replacement for these devices – the C8200-UCPE.
Gonna see if I can't get esxi going on that device and will report back. But if someone's done it already, would love to hear if there were any issues.
eBay of course
Alas, looks like the other one was also snapped up. $225 ea lol, I figured they wouldn't last but I just couldn't justify buying both at the time
Yeah, I looked there and everything was out of my budget range. Completed/Sold hasn't even caught up to those at that price. I'll have to keep my eyes open. Looks like there's been some at $250, worth considering at that price point.eBay of course
ETA: As I'm digging, looks like the C8300-UCPE-1N20 is the one to look for as a replacement to the ENCS5412... Looks like the 8200-UCPE-1N8 is positioned to replace the ENCS5108.
Last edited:
I believe the C8200-UCPE has the Atom C3758 CPU vs the D1557 in the ENCS5412, and Xeon D2xxx in the C8300
I would be curious how the atom performs compared to the ENCS.
Sure wish there were > 1G options for the ENCS. I got a great deal on a C3560-8x24UQ for mGig (eBay) and so far USB hasn't worked with ESXi. I suspect it might be a power draw issue but need to dig into it more.
I would be curious how the atom performs compared to the ENCS.
Sure wish there were > 1G options for the ENCS. I got a great deal on a C3560-8x24UQ for mGig (eBay) and so far USB hasn't worked with ESXi. I suspect it might be a power draw issue but need to dig into it more.
Dang you guys can read better than me... Still, gonna see what I can get running on it. Maybe this weekend. First challenge is I need to find a device I can use to install an os on the drive since there's no oob or video port on the 8200
I managed to get a Realtek 8156 adapter working on the ENCS5412 USB port by using a powered USB hub, under ESXi using the USB NIC fling. VM throughput is right around 2Gbps. It has been stable so far, esxtop shows "system" CPU usage around 60%
I want to test an 8157 (5Gbps) also but the one I have is so flaky on a "normal" host it's not worth testing here - need to find one that doesn't lock up constantly.
I want to test an 8157 (5Gbps) also but the one I have is so flaky on a "normal" host it's not worth testing here - need to find one that doesn't lock up constantly.
This one is kind of a weird in-between... even Cisco's own documentation goes back and forth between assuming you are running IOS-XE and mentioning that it will run NFVIS and 8000v... but I can't find HOW to do it.
You should be able to go in through the console port and manage the box but CIMC console is annoying...
This should be what you need to get into it. I'm curious if they do the "shared LOM" thing on these where you manage the CIMC and the NVFIS through the G0/0 port or something. They definitely make references in the documentation to setting the management port but I can't find exactly how to do it.
This is how you set the CIMC IP on the 5400 at least it might be of help.
scope cimc
scope network
set v4-add x.x.x.x
set v4-netmask 255.255.255.0
set v4-gateway x.x.x.x
commit