Dell VEP/VMWare Edge/Velo Cloud SD-WAN/VeraCloud VEP1400/VEP1400-X firewall units
- Thread starter j_h_o
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Long story short with this seller: Will not send outside US no matter what, will not reply to messages from people outside US no matter what.Nice find! Makes it easy to run a HA-pair in a rack
I respect and understand the former, cannot bear the latter.
Update some findings regarding installs:
There is no way to install Proxmox with serial console. The official way is installing Debian (I choose the netinstall ISO) and from there upgrade to Proxmox: Proxmox through Debian 11 installation
Same problems as Untangle: Going through all the installation without problems only to find that fragging Debian stubbornly wants to install Grub on eMMC instead of SSD and it fails, leaving again a complete installation without form of starting it without troubleshooting that mess. Sorry, I don't have time right now, so next one...
Back to Untangle, because I'm more stubborn than it. Change the approach and deleted the eMMC contents, leaving an empty disk and trying to install on SSD again. And failed again. So it seems debian-based distros have a big problem installing grub on this machine. Also, I don't like the fact that it doesn't lets you choose another disk for installing it. It has to be this way. Period.
Ok then, wiped both disks and installed Untangle on the eMMC. This time it was a success and I can boot it, but leaves the bitterweet taste of not beign able to match OPNSense in terms of ease of installation and configuration.
Right now it's a no-no for me.
Next ones will be Vyos and PFSense.
Here you have the option of creating a config file for the installer with customized defaults, or what i would do, bootstrap debian instead of using the installer, in which case you manually install grub where you need it. Bootstrapping debian also gives you the choice of installing on ZFS root, which then would give you the option of maybe getting zfsbootmenu to work, in case this device supports booting from EFI partition.
That would leave you with a proxmox on ZFS install which you can easily snapshot / clone / boot from clone / rollback from the bootloader.
Either way, at least bootstrapping debian should definitely work here, as long as you find a way to boot a live OS from usb with console access.
Debian root on ZFS:
Debian — OpenZFS documentation
Debian with zfsbootmenu:
Bullseye UEFI — ZFSBootMenu 2.1.0 documentation
Last edited:
To add, there are many more options to consider as well:
If you bootstrap a custom Debian on ZFS root as per the OpenZFS instructions, it will have a different root pool structure compared to a proxmox root on ZFS installed with the proxmox installer. One option would be to install a proxmox on ZFS in a VM, and then create a similar root pool structure manually during the bootstrap. However a bootstrapped debian on ZFS root, with proxmox installed via script results in a proxmox install that doesn't know it's on a ZFS root, which requires alittle more work.
One option is to first get a working Debian on ZFS root on your device, then simply destroy the rpool datasets, and ZFS send-receive the root pool datasets from a VM where you used the proxmox installer.
finally adding zfsbootmenu to the mix, i haven't tried it with proxmox, but ZBM boots from a .EFI image which contains a embedded kernel+initramfs (it's essentially a small linux distro). It searches your debian/proxmox root pool for kernels, and then as you actually boot something, it does a kexec to the new root, so the OS you boot will be totally unaware that ZBM even exists. It requires some more work, as for example any kernel command lines will have to be defined using zfs properties etc.. minor issues because of the way ZBM works..
But to summarize, i assume that you optimally would want a proxmox on ZFS root in the device. What i would do is first install proxmox on ZFS root in a VM using the proxmox installer. Then create a working debian root on ZFS bootstrap on the device, and finally when Debian is booting on your device, destroy the debian root datasets, and zfs send-receive the proxmox datasets from the VM in their place. Making sure also the grub menu entries etc match. It should be possible.
If you bootstrap a custom Debian on ZFS root as per the OpenZFS instructions, it will have a different root pool structure compared to a proxmox root on ZFS installed with the proxmox installer. One option would be to install a proxmox on ZFS in a VM, and then create a similar root pool structure manually during the bootstrap. However a bootstrapped debian on ZFS root, with proxmox installed via script results in a proxmox install that doesn't know it's on a ZFS root, which requires alittle more work.
One option is to first get a working Debian on ZFS root on your device, then simply destroy the rpool datasets, and ZFS send-receive the root pool datasets from a VM where you used the proxmox installer.
finally adding zfsbootmenu to the mix, i haven't tried it with proxmox, but ZBM boots from a .EFI image which contains a embedded kernel+initramfs (it's essentially a small linux distro). It searches your debian/proxmox root pool for kernels, and then as you actually boot something, it does a kexec to the new root, so the OS you boot will be totally unaware that ZBM even exists. It requires some more work, as for example any kernel command lines will have to be defined using zfs properties etc.. minor issues because of the way ZBM works..
But to summarize, i assume that you optimally would want a proxmox on ZFS root in the device. What i would do is first install proxmox on ZFS root in a VM using the proxmox installer. Then create a working debian root on ZFS bootstrap on the device, and finally when Debian is booting on your device, destroy the debian root datasets, and zfs send-receive the proxmox datasets from the VM in their place. Making sure also the grub menu entries etc match. It should be possible.
Last edited:
To add yet more, i used to run a proxmox cluster on 12x Optiplex 780's that all booted over iscsi/ipxe. I had just one proxmox on XFS installed in VM on a single zvol, the zvol was then transferred to a SSD, and then was ZFS cloned into 12x zvol's, with only the minor changes such as hostname for each zvol. That way i could run 12 proxmox nodes that only took the space of a single proxmox install on one zvol.
But to the point, another option here is, assuming this device has even basic PXE boot support, then you could use ipxe with PXE chainloading to boot it off iscsi. I wish i had written an extensive guide how i ran a 12 node proxmox cluster over multipath iscsi on old hardware, but if you need advice, i can dig through old configs and tell you exactly how to do that.
If you run proxmox and need to IOV partition the network interfaces for various VM's, i have a fresh systemd script for that as well, that supports static MAC addresses, plus start stop reload etc..
But to the point, another option here is, assuming this device has even basic PXE boot support, then you could use ipxe with PXE chainloading to boot it off iscsi. I wish i had written an extensive guide how i ran a 12 node proxmox cluster over multipath iscsi on old hardware, but if you need advice, i can dig through old configs and tell you exactly how to do that.
If you run proxmox and need to IOV partition the network interfaces for various VM's, i have a fresh systemd script for that as well, that supports static MAC addresses, plus start stop reload etc..
I haven't had any issues with Debian or CentOS installs, as for Proxmox, I used the Debian-to-Proxmox method, but I use that most of the time anyway to be honest because it fits well in my Packer and Debootstrap methods. As for GRUB and EFI booting: I can always choose the installation location of GRUB, but I do know that it fails to show disks when you boot with CSM enabled since GRUB will think it needs to do some bootsector nonsense while what it really needs to do is drop the grub .efi binary on to a valid ESP. The VEP-X models have eMMC and an SSD and I can boot both from the eMMC and the SSD, and I can also have GRUB on either ESP and have the EFI boot manager boot from any of the ESP located EFI programs.
I did at one time have an issue with CentOS booting, but that turned out to be my own fault for using some very old boot media.
Seeing the success eloich had on a 620. I got all of the interfaces on a 640 functioning in pfsense as well. Hitting the button closest to the ethernet ports was all that it took. Hit that, board goes through a reset a couple of times and then all of the interfaces become functional.
I have a 610 I am playing with as well. Ethernet ports on that run through a Marvel 88E6190 embedded switch. Etherswitch support is not built into the community version of pfsesne. But it is pfsense plus. I'm here so far and will play with it more tomorrow.
Brought network connectivity to the 610 via USB ethernet adapters.
Upgraded CE to pfsense plus.
pfSense on Watchguard M270 had some info on the etherswitch config for the intel C3K SOC talking to a 88E6190.
I've gotten a little further. Need to boot back into the Dell DIAG OS to see how it lays out the configuration for the 88E6190 when talking to the intel SOC ports.
Brought network connectivity to the 610 via USB ethernet adapters.
Upgraded CE to pfsense plus.
pfSense on Watchguard M270 had some info on the etherswitch config for the intel C3K SOC talking to a 88E6190.
I've gotten a little further. Need to boot back into the Dell DIAG OS to see how it lays out the configuration for the 88E6190 when talking to the intel SOC ports.
I am very jealous of all of this. We just recorded a 15x switch round-up video today and I have recorded videos the last two days as well, and am hoping to get 1-2 done tomorrow. I wish I had one / the time to play with one.
Happy that my somewhat old clumsy fingers were actually able to figure out that one. As someone surely said before "when there is a switch, there is a way"! Kidding aside, I may or may not be on the lookout to try further OS'es on that particular unit in the near future, as I'm tempted to put it into production.
I've also been able to replace the diesel sounding fan that was on my unit with a CUI Devices CFM-4010V-150-157-20, paired with a spliced in WM15253 pigtail (5.4CFM 5000RPM) which pushes/pulls a bit less air than the original Delta AFB0412MA-A (5.7CFM, 5000RPM) but nonetheless keeps the unit operating beween 47-52C (ambiant temp is around 20-22C). That's with low to moderate load so YMMV if the system is pushed. I also haven't taken the time to check the CPU heatsink thermal paste but I kind of assume that since the unit may not be more than 2-3 years old it should not have dried up by now... Considering the temperature limits for the C3xxx cpu it's well within the acceptable parameters anyways.
And by the way, on the two non SFP+ ix interfaces: they're indeed locked down somehow, and this is even shown in the related BIOS screen. Nonetheless, (and for my own purposes at least) two 10gb ports are plenty for the intended workload, and furthermore, not sure if the system could sustain 4 simultaneous 10gb streams without huffing and puffing so, in my opinion, it's all good.
Cheers,
Success on the 610 for pfSense Plus!
Add the following to /boot/device.hints: (note: these will not persist past a system update, /boot/loader.conf.local doesn't seam to persist past a reboot on pfSense plus. I plan to look at that later)
hint.mdio.0.at="ix1"
hint.e6000sw.default_disabled=0
hint.e6000sw.0.addr=0
hint.e6000sw.0.is6190=1
hint.e6000sw.0.port0disabled=1
hint.e6000sw.0.port9cpu=1
hint.e6000sw.0.port10cpu=1
hint.e6000sw.0.port9speed=2500
hint.e6000sw.0.port10speed=2500
Created /usr/local/etc/rc.d/setup_switch.sh with the following: (I plan to move this to config.xml using <shellcmd> or <earlyshellcmd>. As it stands, setup_switch.sh is called after all the network init has already been done on ix0 and ix1)
#!/bin/sh
echo "Configuring switch..."
logger Configuring switch...
etherswitchcfg vlangroup1 vlan 1 members 10
etherswitchcfg vlangroup2 vlan 2 members 3,4,5,6,7,8,9
etherswitchcfg vlangroup3 vlan 3 members 2,4,5,6,7,8,9
etherswitchcfg vlangroup4 vlan 4 members 2,3,5,6,7,8,9
etherswitchcfg vlangroup5 vlan 5 members 2,3,4,6,7,8,9
etherswitchcfg vlangroup6 vlan 6 members 2,3,4,5,7,8,9
etherswitchcfg vlangroup7 vlan 7 members 2,3,4,5,6,8,9
etherswitchcfg vlangroup8 vlan 8 members 2,3,4,5,6,7,9
etherswitchcfg vlangroup9 vlan 9 members 2,3,4,5,6,7,8
etherswitchcfg vlangroup10 vlan 10 members 1
etherswitchcfg port1 forwarding
etherswitchcfg port2 forwarding
etherswitchcfg port3 forwarding
etherswitchcfg port4 forwarding
etherswitchcfg port5 forwarding
etherswitchcfg port6 forwarding
etherswitchcfg port7 forwarding
etherswitchcfg port8 forwarding
etherswitchcfg port9 forwarding
etherswitchcfg port10 forwarding
echo "done"
logger done
This ties ix0 to Ge1 on the 610. Then ix1 is tied to Ge2-8
Add the following to /boot/device.hints: (note: these will not persist past a system update, /boot/loader.conf.local doesn't seam to persist past a reboot on pfSense plus. I plan to look at that later)
hint.mdio.0.at="ix1"
hint.e6000sw.default_disabled=0
hint.e6000sw.0.addr=0
hint.e6000sw.0.is6190=1
hint.e6000sw.0.port0disabled=1
hint.e6000sw.0.port9cpu=1
hint.e6000sw.0.port10cpu=1
hint.e6000sw.0.port9speed=2500
hint.e6000sw.0.port10speed=2500
Created /usr/local/etc/rc.d/setup_switch.sh with the following: (I plan to move this to config.xml using <shellcmd> or <earlyshellcmd>. As it stands, setup_switch.sh is called after all the network init has already been done on ix0 and ix1)
#!/bin/sh
echo "Configuring switch..."
logger Configuring switch...
etherswitchcfg vlangroup1 vlan 1 members 10
etherswitchcfg vlangroup2 vlan 2 members 3,4,5,6,7,8,9
etherswitchcfg vlangroup3 vlan 3 members 2,4,5,6,7,8,9
etherswitchcfg vlangroup4 vlan 4 members 2,3,5,6,7,8,9
etherswitchcfg vlangroup5 vlan 5 members 2,3,4,6,7,8,9
etherswitchcfg vlangroup6 vlan 6 members 2,3,4,5,7,8,9
etherswitchcfg vlangroup7 vlan 7 members 2,3,4,5,6,8,9
etherswitchcfg vlangroup8 vlan 8 members 2,3,4,5,6,7,9
etherswitchcfg vlangroup9 vlan 9 members 2,3,4,5,6,7,8
etherswitchcfg vlangroup10 vlan 10 members 1
etherswitchcfg port1 forwarding
etherswitchcfg port2 forwarding
etherswitchcfg port3 forwarding
etherswitchcfg port4 forwarding
etherswitchcfg port5 forwarding
etherswitchcfg port6 forwarding
etherswitchcfg port7 forwarding
etherswitchcfg port8 forwarding
etherswitchcfg port9 forwarding
etherswitchcfg port10 forwarding
echo "done"
logger done
This ties ix0 to Ge1 on the 610. Then ix1 is tied to Ge2-8
Yep, I think OpenWRT has the most up-to-date information Ethernet Network Switch
And since it's native to linux, you can probably setup the switch on the VyOS shell first, and then consume the interfaces in the VyOS cli tools.
And since it's native to linux, you can probably setup the switch on the VyOS shell first, and then consume the interfaces in the VyOS cli tools.
Time to report back my findings so far:
After installing Untangle on the eMMC I wanted to install DIAG-OS again on it.
Weird thing that happened: tried more than 7 times and all failed because GRUB cannot install in MBR mode. So it seems I won't burn properly an ISO in RUFUS until it's the 8th time in a row.
Just after installing DIAG-OS went into BIOS and disabled CSM Support.
Then, surprisingly, I was able to install Debian 11 right off the bat with the only hiccup of having to repeat the ethernet configuration multiple times until I found the eth interface it was using. This time GRUB installed on the SSD without having to select anything and even the SSH server worked, starting a session through PuttY on first try.
Next step was installing Proxmox following their guide. All worked until the reboot after the pve-kernel-5.15 installation.
If I selected the debian kernel it booted fine, but choosing the pve-kernel meant that console was flooded by error "acpi fpdt: duplicate resume performance record found" which looped ad eternum.
The only reference I could find was an answer in Proxmox forum's from October 2022 stating that a BIOS downgrade to the original one fixes this problem.
Got nothing to loose so there I went and, indeed, it fixed the problem. Maybe this approach is too radical, but didn't find anything better.
Right now I have a Proxmox installation that works, with the gui available, ssh enabled and fully functional.
Next thing is to upgrade BIOS to last version and check if error comes back.
P.S.: Tinkering with this appliance gives me the feeling of running 110 hurdles on groundhog day.
After installing Untangle on the eMMC I wanted to install DIAG-OS again on it.
Weird thing that happened: tried more than 7 times and all failed because GRUB cannot install in MBR mode. So it seems I won't burn properly an ISO in RUFUS until it's the 8th time in a row.
Just after installing DIAG-OS went into BIOS and disabled CSM Support.
Then, surprisingly, I was able to install Debian 11 right off the bat with the only hiccup of having to repeat the ethernet configuration multiple times until I found the eth interface it was using. This time GRUB installed on the SSD without having to select anything and even the SSH server worked, starting a session through PuttY on first try.
Next step was installing Proxmox following their guide. All worked until the reboot after the pve-kernel-5.15 installation.
If I selected the debian kernel it booted fine, but choosing the pve-kernel meant that console was flooded by error "acpi fpdt: duplicate resume performance record found" which looped ad eternum.
The only reference I could find was an answer in Proxmox forum's from October 2022 stating that a BIOS downgrade to the original one fixes this problem.
Got nothing to loose so there I went and, indeed, it fixed the problem. Maybe this approach is too radical, but didn't find anything better.
Right now I have a Proxmox installation that works, with the gui available, ssh enabled and fully functional.
Next thing is to upgrade BIOS to last version and check if error comes back.
P.S.: Tinkering with this appliance gives me the feeling of running 110 hurdles on groundhog day.
What package did you use to update the BIOS on your 610? Would you mind linking to what you used, if any?
Hi there,
nice thread! I’m contemplating getting an Edge 620 as a 10GbE router for my home internet, and I have a couple questions:
1. How loud is the fan? Is it compatible with “significant others” in a living room? Can it be replaced with a quieter one?
2. Can I use the firmware upgrades from the Dell site for any version of the 6x0 series, regardless of brand (VeloCloud, VMWare, etc)?
3. Can it actually do 10GbE routing with just NAT?
Thanks a lot!
nice thread! I’m contemplating getting an Edge 620 as a 10GbE router for my home internet, and I have a couple questions:
1. How loud is the fan? Is it compatible with “significant others” in a living room? Can it be replaced with a quieter one?
2. Can I use the firmware upgrades from the Dell site for any version of the 6x0 series, regardless of brand (VeloCloud, VMWare, etc)?
3. Can it actually do 10GbE routing with just NAT?
Thanks a lot!
I'll answer the questions for which I know the answers.1. How loud is the fan? Is it compatible with “significant others” in a living room? Can it be replaced with a quieter one?
2. Can I use the firmware upgrades from the Dell site for any version of the 6x0 series, regardless of brand (VeloCloud, VMWare, etc)?
3. Can it actually do 10GbE routing with just NAT?
1. Fan is (or should be) almost silent (if the unit has been operating in a normal environment prior). The unit I received was louder because the fan was abnormally loud as it was defective (had some kind of metallic powder stuck on it, part of which had made its way in the bearing assembly). Replacement fan with similar CFM can't be heard unless you put you ear inches away from it.
2. For the Edge 620 (same as the one I have), I've installed the VEP1400-X firmware. Note: haven't tried on any other brand but if it's the same board, it should work. Got the firmware from Dell at this location.
For #3, can't give you anything else but my opinion; might not be able to push 10Gb using *bsd, but maybe with other OSes. Again, this is only my opinion so YMMV. There are other knowledgeable people that may be able to answer this question way more accurately on this forum.
Cheers,
Pushing 10G with on OpnSense works at least in my setups were it's mainly just inter-VLAN routing and firewalling (2x DAC between switch and VEP). There are a few setups where we are pushing 2Gb NAT because that just happens to be the WAN speed, don't really have them deployed for anything faster. (and at the point where someone can pay for fast internet, they can also just pay for fast dedicated hardware IMO so never had that issue)
As for fan and firmware: can confirm experience, and since Dell apparently churns out all kinds of C3000 white label boxes it makes sense it has a common firmware base. The most significant difference seems to be the network topology, but as soon as you get 10G SFP+ connectivity the firmware is the same as the VEP-X.
As for fan and firmware: can confirm experience, and since Dell apparently churns out all kinds of C3000 white label boxes it makes sense it has a common firmware base. The most significant difference seems to be the network topology, but as soon as you get 10G SFP+ connectivity the firmware is the same as the VEP-X.
Last edited: