Dell VEP/VMWare Edge/Velo Cloud SD-WAN/VeraCloud VEP1400/VEP1400-X firewall units

[blunden]

You will find lots of info stating that only the paid version of pfsense supports the QAT - this is not true!


It turns out that within your OpenVPN instance you must set the instance type to DCO.

You are making the incorrect assumption that pfSense and OPNsense is essentially the same thing. That's not true.

The paid version of pfSense sometimes have some features and performance improvements in the paid version that aren't available for OPNsense or the free version of pfSense at all. OPNsense also sometimes give you some features for free that are limited to the paid version of pfSense. QAT is one of the latter features, as far as I know. In other words, it's true that it's a paid feature in pfSense.

Yes, OpenVPN DCO + hardware acceleration is the only way to get good performance from OpenVPN.

 

[ccie4526]

I am still looking for a way to control the fan speeds when running OPNsense on the 640. This is the only thing holding me back from deploying it in my home lab. I have considered running Proxmox on the 640, with OPNsense as a VM, but would like to eliminate the extra dependency.

On my 680, I simply replaced the existing fans with Noctua units. I was never able to source a cable with the connector for the 680 mainboard, so I had to cut and splice the wires to the Noctua fan. On my 640, I just disconnected the single fan and have been monitoring temps, and they have not been excessive. YMMV on that though, depending on local environmentals. As I'm not virtualizing, just pfS native, it's not hammering the CPU all that hard, and thus not generating that much heat.

 

[WN1X]

On my 680, I simply replaced the existing fans with Noctua units. I was never able to source a cable with the connector for the 680 mainboard, so I had to cut and splice the wires to the Noctua fan. On my 640, I just disconnected the single fan and have been monitoring temps, and they have not been excessive. YMMV on that though, depending on local environmentals. As I'm not virtualizing, just pfS native, it's not hammering the CPU all that hard, and thus not generating that much heat.

Interesting...my 640 has two fans. It seems to run ok with both disconnected.

 

[rory]

You are making the incorrect assumption that pfSense and OPNsense is essentially the same thing. That's not true.

I made no such assumption at all, nor was I comparing the two products in any way. My point is that the documentation on how to make QAT function on OPNsense is practically non-existant. Enabling QAT in the settings gives no indication that you must also enable DCO, nor does enabling DCO mention that it will utilize QAT if hardware acceleration is set to QAT. There are no examples in the OPNsense documentation on how to configure this, and no hints in the online help.

A Google search of "OPNsense OpenVPN QAT configuration" will yield incorrect AI summaries and references to posts by people claiming that it does not work in OPNsense and that it only works in the paid version of pfSense, and the whole point of my post is to document this as false and to provide the missing steps (e.g. you must enable hardware acceleration = QAT AND configure the instance type to DCO). Failure to perform both will result in a configuration that functions but does not utilize QAT.

Again, perhaps this is obvious to others, but it certainly was not to me. The lack of example or documentation lead me to try and configure unneeded dev nodes for crypto and consider switching to wireguard - which would involve reconfiguring all the clients.

 

[blunden]

I made no such assumption at all, nor was I comparing the two products in any way. My point is that the documentation on how to make QAT function on OPNsense is practically non-existant. Enabling QAT in the settings gives no indication that you must also enable DCO, nor does enabling DCO mention that it will utilize QAT if hardware acceleration is set to QAT. There are no examples in the OPNsense documentation on how to configure this, and no hints in the online help.

A Google search of "OPNsense OpenVPN QAT configuration" will yield incorrect AI summaries and references to posts by people claiming that it does not work in OPNsense and that it only works in the paid version of pfSense, and the whole point of my post is to document this as false and to provide the missing steps (e.g. you must enable hardware acceleration = QAT AND configure the instance type to DCO). Failure to perform both will result in a configuration that functions but does not utilize QAT.

Again, perhaps this is obvious to others, but it certainly was not to me. The lack of example or documentation lead me to try and configure unneeded dev nodes for crypto and consider switching to wireguard - which would involve reconfiguring all the clients.

Ooh, I totally misinterpreted the first part of the post I quoted. I thought you meant it was included in pfSense CE too. You are certainly right about its availability in OPNsense.

The part about incorrect AI summaries makes sense and it's definitely a real issue. Whether that's because they are basing the response on outdated information or whether it's because it assumes that pfSense and OPNsense are the same.

I wouldn't say it's obvious, no. The documentation could absolutely use some improvements in this area. I just agreed with you that DCO is generally important for OpenVPN performance as it removes a bunch of overhead due to context switching.

 

[rory]

On a whim I was able to pick up a 3400 for under $100 to play with. I know the onboard SSD slots are not NVME, but has anyone tried to use the same strategy to add an NVMe drive to these that is used for the 620-680? It looks like you can pick up the WiFi add in card easily (at least one seller on ebay has over 120 in stock - I imagine you can offer $25-$30) and convert the mini nvme slot to a m.2 slot. Has this been tried? Does anyone know the number of pcie lanes on this mini pcie slot?

 

[Arnaud]

Funny, I had the exact same thought as you with the nvme expansion. As it is the same Wifi card as the one in the VEP1400 (WLE600VX) it is likely that only one pcie lane is wired to the minipcie slot.
Now since the rNDC mezzanines expose a pcie 3.016x slot, the unicorn would be finding a carrier that supports 2-4 NVME on an OCP2.0 slot (AFAIR the bios supports bifurcation, I'll try to check later today)

Found under Socket Configuration

 

[shor0814]

On a whim I was able to pick up a 3400 for under $100 to play with. I know the onboard SSD slots are not NVME, but has anyone tried to use the same strategy to add an NVMe drive to these that is used for the 620-680? It looks like you can pick up the WiFi add in card easily (at least one seller on ebay has over 120 in stock - I imagine you can offer $25-$30) and convert the mini nvme slot to a m.2 slot. Has this been tried? Does anyone know the number of pcie lanes on this mini pcie slot?

Have you verified the installation of other OSs or Proxmox? I have 4 of the vep4600 and they carry a premium because allegedly these 3400 versions have watchdog timers, etc and wont work with other operating systems so i have hesitated.

These would be outstanding entry level routers for TNSR or VyOS or even PF/OPNsense. The interposer slots are super flexible. Might even work well with SONiC.

 

[Arnaud]

AFAIK the watchdog problem was only for some specific bios versions of the 600 series, not 3400.

 

[shor0814]

AFAIK the watchdog problem was only for some specific bios versions of the 600 series, not 3400.

That's good to know, the 3400 is (was?) significantly easier and cheaper to get than the 4600 with similar specs.

If these run generic Linux unmodified they can be a great option.

 

[rory]

That's good to know, the 3400 is (was?) significantly easier and cheaper to get than the 4600 with similar specs.

If these run generic Linux unmodified they can be a great option.

If it helps, the 3400 I just bought is up and running ESXi and OPNsense. I have not personally tried linux yet, but the DiagOS image I used to upgrade the firmware is Linux based and appears to work fine.

 

[rory]

Funny, I had the exact same thought as you with the nvme expansion. As it is the same Wifi card as the one in the VEP1400 (WLE600VX) it is likely that only one pcie lane is wired to the minipcie slot.
Now since the rNDC mezzanines expose a pcie 3.016x slot, the unicorn would be finding a carrier that supports 2-4 NVME on an OCP2.0 slot (AFAIR the bios supports bifurcation, I'll try to check later today)

I was ahead of you on this! Unfortunately as far as I can tell there are only three close offerings:

1) There is an OCP 3.0 card for M.2 NVMe drives! Unfortunately this is an entirely different connector and totally incompatible with the OCP 2.0 cards that our mezzanines use. But it validates the concept.

2) There are cards that allow OCP 2.0 cards to be adapted to standard PCIe slots.

3) There is an open source project ocp2pcie on github for item 2 (sort of the inverse of what we want, and mostly in non-english).

Given the three points above, I may try to make one (though I don't have much free time) given that this is only PCIe 3.0 and short trace lengths, it should be fairly simple and totally passive. The ocp2pcie project should be a great starting point!

Let me know if you find anything

 

[Yet Another BugHunter]

By the way, here is a screenshot of PfSense console after booting up:

*** Welcome to Netgate pfSense Plus 23.01-RELEASE (amd64) on pfSense ***

WAN (wan)       -> ix0        -> v4/DHCP4: 192.168.2.186/24
LAN (lan)       -> ix1        -> v4: 192.168.1.1/24

0) Logout (SSH only)                  9) pfTop
1) Assign Interfaces                 10) Filter Logs
2) Set interface(s) IP address       11) Restart webConfigurator
3) Reset webConfigurator password    12) PHP shell + Netgate pfSense Plus tools
4) Reset to factory defaults         13) Update from console
5) Reboot system                     14) Enable Secure Shell (sshd)
6) Halt system                       15) Restore recent configuration
7) Ping host                         16) Restart PHP-FPM
8) Shell

Enter an option:

[/QUOTE]

I have a 620 model with the same BIOS version and the latest memstick version of pfSense CE installed and I've having issues with the serial console. Unless I manually cycle the console value in the grub boot screen (option 5 of the boot menu) the device hangs when it tries to boot. Once booted, after cycling the console value, the pfSense console menu is not displayed, but the UI is accessible via the browser.

Did you make changes to the loader.config file? Or other changes to make boot consistently and the pfSense console menu to display?

Thanks.

 

[ccie4526]

I have a 620 model with the same BIOS version and the latest memstick version of pfSense CE installed and I've having issues with the serial console. Unless I manually cycle the console value in the grub boot screen (option 5 of the boot menu) the device hangs when it tries to boot. Once booted, after cycling the console value, the pfSense console menu is not displayed, but the UI is accessible via the browser.

This really sounds like you have a VGA version of the memstick installer. There is a difference between the "memstick" and "memstick-serial" downloadable images from netgate. Probably check your installer image?

 

[Yet Another BugHunter]

This really sounds like you have a VGA version of the memstick installer.

Thanks for the reply.

I only see one memstick version on the pfSense CE Download site. Is there another place that the non-VGA version can be downloaded from?

The below message from the same site seems to imply the Memstick USB version is the version to install.

The Netgate® Installer is available for:

• Netgate AMD64 appliances with Serial Console (Netgate 1537, 1541, 4100, 4200, 5100, 6100, 7100, 8200, All Other Intel/AMD 64-bit)
• AMD64 ISO IPMI/Virtual Machines
• Netgate AArch64 (ARM64) appliances (Netgate 1100 and 2100)

 

[ccie4526]

I only see one memstick version on the pfSense CE Download site. Is there another place that the non-VGA version can be downloaded from?
The below message from the same site seems to imply the Memstick USB version is the version to install.

Yeah, you can't use their main page for that, you have to go to their download mirror. See NetGate ATXFiles server, get the 2.7.2 memstick-serial. Use that to install, then upgrade to 2.8.1 once you get the 2.7.2 unit running.

 

[Yet Another BugHunter]

Thanks! That did worked.

 

[dagio]

Good and Bad news... my CH341A finally arrived... however in trying to dump the bios it seems that I managed to brick the 610... I just have a solid read light at the back and sfp1 rightside led green... they stay on for around 10 seconds and then turn off for 3 seconds and it just repeats...

Did anyone else have any luck with obtaining a bios image for the edge 610 / VEP1400

******UPDATE*****

There is a second Bios chip on the back. So everyone who bricked their bios by flashing the VEP1400-X bios... you can revert to the previos bios by grabing a dump of the rear bios and all of your data should be retained...

View attachment 28863

these dip switches migh even be a way to enable the rear bios chip... further investigations on going...

Even after turning the switches I have the same red light issue.

 

[Arnaud]

Did you flash using a dump your previously took (potentially corrupted) or a fresh one from the bios update package ?
Also try to reset the cmos by removing the CR2032 battery for like 5 minutes.

If I remember well the Slot 1 for the bios is the one next to the mini-pcie slot, and the Slot 2 is on the bottom of the motherboard, but you should flash both to be sure. The system is quite robust, you should at least be able to see a POST even if the rest is messed up.

 

[dagio]

Did you flash using a dump your previously took (potentially corrupted) or a fresh one from the bios update package ?
Also try to reset the cmos by removing the CR2032 battery for like 5 minutes.

If I remember well the Slot 1 for the bios is the one next to the mini-pcie slot, and the Slot 2 is on the bottom of the motherboard, but you should flash both to be sure. The system is quite robust, you should at least be able to see a POST even if the rest is messed up.

I tried doing all these still i don't see any serial message on my console it is just blinking red btw i tried to flash the primary bios from the recovery bios and then something happened.