You are absolutely awesome! I have seen snippets of this setup elsewhere but never a complete guide or reference. Now to bring the appliances back to the bench!
$127 Cisco ENCS5412/K9 Xeon-D 1557 (12 core), 32G ram
- Thread starter RobstarUSA
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Does anyone have access to the latest NFVIS updates they could share privately? https://software.cisco.com/download/home/286308693/type/286309317/release/4.15.5
It's been a long thread and after catching up, it seems that NFVIS isn't really necessary? I seems that some people installed baremetal linux and some loaded Cisco branded ESXi 8? But NFVIS has some magic stuff that makes the switch works? (Although I don't really plan on using this for routing.)
My docker server just went offline this morning and I can't power it back up again. Power supply probably went bad. So it's time to upgrade to this bad boy.
It's been a long thread and after catching up, it seems that NFVIS isn't really necessary? I seems that some people installed baremetal linux and some loaded Cisco branded ESXi 8? But NFVIS has some magic stuff that makes the switch works? (Although I don't really plan on using this for routing.)
My docker server just went offline this morning and I can't power it back up again. Power supply probably went bad. So it's time to upgrade to this bad boy.
I loaded the Cisco ESXI 7.0.3 image on mine, just not using the 8 switchports.
Gotta be careful with loading the newer NFVIS, images, they will update (and lock down) the BIOS where you can't F2 into it during bootup, you can only do BIOS stuff through the CIMC interface.
Check a certain cios dot dhitechnical site for the latest NFVIS.
Gotta be careful with loading the newer NFVIS, images, they will update (and lock down) the BIOS where you can't F2 into it during bootup, you can only do BIOS stuff through the CIMC interface.
Check a certain cios dot dhitechnical site for the latest NFVIS.
It's possible to roll back, I did it with mine.... would that I could remember how I did it now.... I've had it in a co-lo for the last two years, so that gives you an idea how long ago I did it. There may still be notes in this thread on how to do it.
You can downgrade. Surprisingly Cisco makes it fairly easy to switch back and forth between versions. The one caveat is that while you can upgrade without re-imaging the NFVIS you will have to wipe the system clean to downgrade. Also downgrading will not back down the CIMC and BIOS versions but they SHOULD be able to be downgraded via the CIMC but I've not tested this a whole lot. Also any NFVIS version after 4.15.X will disable the Marvell switch completely. Everything else still seems to work fine but you'll loose the switch entirely... Even though at one point cisco had the newer software (Up to 4.18.2) on the download page specifically for the 5400 ENCS. Don't ask how I found THAT little tidbit out.
I've been using proxmox on mine since I got it. We use NFVISon the systems I have at my work (including the replacement 8300-UCPE chassis) and I HATE IT. I describe it as cisco handed KVM and linux to a bunch of engineers and said "Make this unusable... and don't document anything about how it's supposed to work."
That being said you are correct you will loose the 8 port switch module and any access to the NIM Modules, but you gain full access of the management port, both of the built in SFP/RJ45 ports (you can only use one media type or the other and it switches to the SFP if one is inserted). You'll also see 2 x710 and 1 x552 10G port, the x710's are wired to the switch (if memory serves) and I have NO clue what the x552 is intended for but it's a backplane connection of some sort.
Proxmox has been VERY stable for me. I've had it up and running for a couple years with no issues. fan throttling seems to work ok and if you upgrade the ram (I went with 64G total) you end up with a pretty capable platform for hosting routing VMs, docker hosts and whatever storage based servers (NAS/nextcloud/immich etc). The lack of any real functional video card makes it a little weak for hosting a media server like plex/jellyfin unless you aren't planning on doing any transcoding etc. The storage is somewhat limited with one internal m.2 and 2 2.5" drive bays. If I remember correctly too there's a size limitation on the 2.5" I think it's a phsyical thing though like only 10mm or thinner. The M.2 slot is a regular NVME type not a SATA but due to the placement and cooling I would be careful what you put in there. I generally use mine as a boot drive for Proxmox and an internal storage for ISOs and Images etc. I have 2x 1TB HDDs installed in the 2.5" bays configured as a ZFS pool under proxmox there is an optional hardware raid card available that mine didn't come with but honestly for proxmox it's not necessary since ZFS prefers to manage the individual drives directly anyway. I can't attest to anything VMWare but I would assume the capabilities aren't much different. I don't see Cisco providing VMware/Broadcom with any of the proprietary drivers or software (and from what others on here have reported they didn't) That being said I'd have to see some kind of side by side comparison to prove that there is any benefit to one platform or another. Although Turbo did have some better luck than I did with getting an NFVIS VM to boot and recognize the switch module. Which may or may not be of interest. The other method described in this thread works but it's FAR from simple and definitely doesn't end up with anything I'd call convenient for configuration and management of the switch.