Dell VEP/VMWare Edge/Velo Cloud SD-WAN/VeraCloud VEP1400/VEP1400-X firewall units

[JaY_III]

As requested. A little update: I was able to get opnsense installed on the eMMC. That went without a hitch by following the documentation.

I then switched to proxmox, which I'm more familiar with. Proxmox does not allow installing on eMMC as a default option. You have to install in debug mode. I actually was not able to get this working though as going into the advanced install menu under the serial console seems to have a bug. It kicks you out to the normal install menu whenever you use the arrow keys. I didn't try resolving this and just installed on the SSD as I'm still very much in testing stages.

My biggest gripe so far is the fan noise, which PVE/linux looks like an easier path to managing than BSD/OPNsense. I may do a hardware mod as well with some arctics or noctuas but not really interested in investing much more time or resources into this side project at the moment. A few starting hiccups aside, nice little unit for homelab uses.


View attachment 49019

Just an FYI if you want to do the fan mod....
The unit ships with nice (but load) Deltas, and the common just swap them for Noctuas is not as straight forward as you would think.
The board seems to monitor the current draw of the fan and if not enough is being pulled it wont spin the fans at all.

My system would initially spin the noctuas up then after a short while they would turn off. You need to add a resistor in parallel to trick the fan controller..... As I thought that was a silly solution, what I ended up doing was soldering the 12V leads to the back side of the 12V input power on the board, pulling power directly from the PSU.

They hack works well with one minor annoyance. When you soft power off the system, the fans keep spinning so they no longer tell you when the system is fully shutdown. Minor and easy to live with,

for reference temps used to sit around 45Cusing opensense and now with the silent mod, i am around 51C.

Bit of a pain to get done, but i cant hear it and my ears thank me.

 

[nmpu]

Just an FYI if you want to do the fan mod....
The unit ships with nice (but load) Deltas, and the common just swap them for Noctuas is not as straight forward as you would think.
The board seems to monitor the current draw of the fan and if not enough is being pulled it wont spin the fans at all.

My system would initially spin the noctuas up

The device is capable of monitoring the RPM. The fan controller has thresholds. That's probably what needs to change. If you're lucky, you could just program the relevant registers via I2C.

 

[JaY_III]

The device is capable of monitoring the RPM. The fan controller has thresholds. That's probably what needs to change. If you're lucky, you could just program the relevant registers via I2C.

I could not get them to stay spinning without adding resistor between the ground and 12V.
I even tried two fans (1 RPM sensor) on a single header with no luck and got tired of fighting whatever the fan controller is programed to do.
So i picked 12V and no longer have any issues.

 

[thorazine74]

Hello, I got a white vmware 620 unit that I plugged in a 19V laptop psu by mistake. I assumed that it was fully damaged but when I plug in the right 12v psu it actually turns on, the front led turns on in red, the fan turns on and the little tiny led on the board (that is only visible if you remove the case) turns on steady green. After maybe 8-10 seconds it reboots itself and the same sequence start over again. No output at over serial except some individual chars (noise?)
Is it normal that the unit does this? I would have assumed I completely fried it when I fed it 19V and it would not power on at all.
Is there anything it can be done software wise to recover from this?

 

[nmpu]

I plugged in a 19V laptop psu by mistake.

Only 19V? I did the same thing with a 640 and a 48V POE power supply. Too many barrel plugs under my desk.

I've got the same reset sequence. These devices have multiple power management ICs. Something did get fried. There is no software fix. Even if you knew which chip(s) were damaged and could remove and resolder the fine-pitch components, they'd likely still require custom programming. Could both BIOS slots have been corrupted? Maybe. I got my devices so cheap that troubleshooting is not a priority.

For anyone messing around inside, be careful with the SATA and mini PCIe slots (and unpopulated B-key slot). According to Dell, those are not user-serviceable. They probably lack overload protection and may not be 100% standard pinout. You can end up with the same power-cycle sequence-- permanently.

 

[Halken]

I am about to pull the trigger on a 640, that I plan to install Opnsense on and use as 10g router. I am a bit of a noob on both. Anything I should be obs on?
Was there also possible to enable to 10gb lan on the non-SFP+ ports? Not that I will tax them the same time but its nice to have the option, depending on what ones inet sevice is in the future.

 

[nmpu]

@Halken Is it a black VEP1445 or the white vmware model? There can be subtle differences in how much memory is included, SATA drive size, and whether there's a mini PCIe slot with Wi-Fi. Best case for a stock 640 is 32GB memory, 256GB SSD, and Wi-Fi. Realize that you won't actually achieve 10Gb throughput. Routing is highly dependent on single-core speed. The C3758 is relatively slow.

Most people want to update the BIOS and other firmware. This can be problematic, but 99% are successful. I've updated 10 units and never had to resort to manual FLASH programming like the few horror stories you may read.

I'm in the US and have purchased a scratched up, but fully functional 640 (no PS or accessories) for as low as $38 with shipping. I see you're in the EU, so your options may be limited. How much are you planning on spending? Maybe someone here can offer you a better deal.

 

[blunden]

I am about to pull the trigger on a 640, that I plan to install Opnsense on and use as 10g router. I am a bit of a noob on both. Anything I should be obs on?
Was there also possible to enable to 10gb lan on the non-SFP+ ports? Not that I will tax them the same time but its nice to have the option, depending on what ones inet sevice is in the future.

No, only the SFP+ ports are 10 Gbit capable as far as I understand it. The Atom C3758 found in some of these does have 4 x 10 Gbit NICs built-in, but it seems like two of them are connected to 1 Gbit PHYs according to early posts in this thread.

Hopefully someone else knows for sure though.

Realize that you won't actually achieve 10Gb throughput. Routing is highly dependent on single-core speed. The C3758 is relatively slow.

It won't give him single stream/connection throughput of 10 Gbit/s, no. With a few simultaneous streams/connections, it will, at least on Linux.

 

[Halken]

@Halken Is it a black VEP1445 or the white vmware model? There can be subtle differences in how much memory is included, SATA drive size, and whether there's a mini PCIe slot with Wi-Fi. Best case for a stock 640 is 32GB memory, 256GB SSD, and Wi-Fi. Realize that you won't actually achieve 10Gb throughput. Routing is highly dependent on single-core speed. The C3758 is relatively slow.

Most people want to update the BIOS and other firmware. This can be problematic, but 99% are successful. I've updated 10 units and never had to resort to manual FLASH programming like the few horror stories you may read.

I'm in the US and have purchased a scratched up, but fully functional 640 (no PS or accessories) for as low as $38 with shipping. I see you're in the EU, so your options may be limited. How much are you planning on spending? Maybe someone here can offer you a better deal.

Its mostly black Dell EMC 640 models that are avalible in EUr. WiFi is quite important.

Good to hear. As far as I understand, the 640 does not have the BIOS timer, so less of an issue.
I can see they sell for 300-400€ here, so way more than in the US. But as my router-AP is a TPLink, which is compromised, I am looking to replace it with this running opnsense. Wifi should be standard if its not a N model?
That would be nice.

I have looked at a 16 core 680, but that will not help, if I understand it right, as it would be the same slow cores....

 

[Halken]

It won't give him single stream/connection throughput of 10 Gbit/s, no. With a few simultaneous streams/connections, it will, at least on Linux.

Thx for clearing that up. Its 1g now, and current there is no plan on changing it. I was looking to run BSD/opnsense on it. 10gb would be future proofing it. 2.5-5g is likely where it goes next.

 

[blunden]

Its mostly black Dell EMC 640 models that are avalible in EUr. WiFi is quite important.

Good to hear. As far as I understand, the 640 does not have the BIOS timer, so less of an issue.
I can see they sell for 300-400€ here, so way more than in the US. But as my router-AP is a TPLink, which is compromised, I am looking to replace it with this running opnsense. Wifi should be standard if its not a N model?
That would be nice.

I have looked at a 16 core 680, but that will not help, if I understand it right, as it would be the same slow cores....

That sounds expensive to me. If you find a cheap one in the US, even with shipping and VAT you'll probably still save quite a bit. When I bought my Intel X710 NIC, I imported one from eBay US and even though the shipping and VAT almost doubled the final price, I still got a good deal.

Is the WiFi chip in these even good in AP mode? Most M.2 WiFi NICs meant for laptops are not necessarily that good as APs.

FreeBSD is also very outdated in terms of WiFi driver and standard support last time I checked. I would definitely recommend something Linux based for WiFi.

Can't you just replace the stock firmware on your TP-Link with OpenWrt and use it either as a router + AP or just an AP?

 

[blunden]

These VEP units still seem like a great value for homelab and routing projects, but it's good to know the limitations around 10G throughput and the fan controller before diving in.

For the low prices they can sometimes be found in the US, they're amazing value, yes.

Some people might not even notice the limitation since many use cases make use of multiple connections. Like you said, it's always good to know such things ahead of time though. If someone is able to get VPP working on them, that might reduce or even completely eliminate this performance limitation.

 

[loldongs]

I was ahead of you on this! Unfortunately as far as I can tell there are only three close offerings:

1) There is an OCP 3.0 card for M.2 NVMe drives! Unfortunately this is an entirely different connector and totally incompatible with the OCP 2.0 cards that our mezzanines use. But it validates the concept.

2) There are cards that allow OCP 2.0 cards to be adapted to standard PCIe slots.

3) There is an open source project ocp2pcie on github for item 2 (sort of the inverse of what we want, and mostly in non-english).

Given the three points above, I may try to make one (though I don't have much free time) given that this is only PCIe 3.0 and short trace lengths, it should be fairly simple and totally passive. The ocp2pcie project should be a great starting point!

Let me know if you find anything

I think the Gigabyte CNVO124 (https://www.ebay.com/itm/177913940835) might just be what you're looking for. It looks to be OCP 2.0 compatible and breaks out into four U.2 NVMe channels.

Alternately, if you want to do SAS a bit cheaper, the ZT System T2618502 (https://www.ebay.com/itm/326444235652) is an LSI SAS2308 in an OCP 2.0 form factor, supporting eight drives across two channels. I think you'd need to find a way to power the drives, but it could work for a couple SSDs or something.

 

[loldongs]

On a whim I was able to pick up a 3400 for under $100 to play with. I know the onboard SSD slots are not NVME, but has anyone tried to use the same strategy to add an NVMe drive to these that is used for the 620-680? It looks like you can pick up the WiFi add in card easily (at least one seller on ebay has over 120 in stock - I imagine you can offer $25-$30) and convert the mini nvme slot to a m.2 slot. Has this been tried? Does anyone know the number of pcie lanes on this mini pcie slot?

I just ordered a 3400 myself for $250 shipped, it should be here this weekend. Excited to get it in the rack and OPNsense installed.

 

[nmpu]

Does anyone know the number of pcie lanes on this mini pcie slot?

The mini PCIe standard supports at most one lane. That means 0 or 1.

 

[Halken]

That sounds expensive to me. If you find a cheap one in the US, even with shipping and VAT you'll probably still save quite a bit. When I bought my Intel X710 NIC, I imported one from eBay US and even though the shipping and VAT almost doubled the final price, I still got a good deal.

Is the WiFi chip in these even good in AP mode? Most M.2 WiFi NICs meant for laptops are not necessarily that good as APs.

FreeBSD is also very outdated in terms of WiFi driver and standard support last time I checked. I would definitely recommend something Linux based for WiFi.

Can't you just replace the stock firmware on your TP-Link with OpenWrt and use it either as a router + AP or just an AP?

Unfortunately not. I will have to replace the router-AP at some point. I am looking at other possibilities, like the openWRT Zyxel T-56.
openWRT is Linux-baset, and I don't need excellent wifi, just that it works in my condo, which is 70,m2. Can openWRT installed on a 640 do that?
Its really about what I can find one for here in Europe vs how many of the boxes it checks.

 

[blunden]

Unfortunately not. I will have to replace the router-AP at some point. I am looking at other possibilities, like the openWRT Zyxel T-56.
openWRT is Linux-baset, and I don't need excellent wifi, just that it works in my condo, which is 70,m2. Can openWRT installed on a 640 do that?
Its really about what I can find one for here in Europe vs how many of the boxes it checks.

I see.

Yes, OpenWrt is Linux-based and the x86-64 build could presumably run on the 640, but I haven't tried it. The fact still remains that client WiFi NICs aren't great at being used as APs in most cases. Is it good enough? Possibly, but I don't know for sure. You can probably find people online who have tried using that particular WiFi NIC in AP mode (in a different device).

 

[matt0611]

Hello, I'm new to these machines. I have a brand new 640 and I'm stuck on trying to get DiagOS installed (either on the emmc or ssd). I keep getting this grub errors. Would someone be able to assist with what I'm doing wrong? Thanks!

Mounted /dev/mmcblk0p1 on /tmp/tmp.cXPbp4

Installing grub for diag-os
ERROR: grub-install failed on: /dev/mmcblk0
Installing for i386-pc platform.
grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible.
grub-install: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
grub-install: error: will not proceed with blocklists.
Removing /tmp/tmp.KgKs3q
Failure: Unable to install image: /diag-installer-x86_64-dellemc_vep1400_c3538-r0-3.43.3.81-26-2022-03-01.bin
This should be not reachable unless something wrong is there!!!!!

EDIT. OK I was able to get passed this by making the USB drive GPT / UEFI only. But am now hitting this error:

./VEP1400-X-BIOS-3.48.0.9-19.bin
-bash: ./VEP1400-X-BIOS-3.48.0.9-19.bin: cannot execute binary file: Exec format error

 

[nmpu]

OK I was able to get passed this by making the USB drive GPT / UEFI only. But am now hitting this error:

Command should be vep1400x_ufw_2.6. Maybe you're following some old instructions? The latest Dell script automatically selects the correct .bin file.

 

[matt0611]

Command should be vep1400x_ufw_2.6. Maybe you're following some old instructions? The latest Dell script automatically selects the correct .bin file.

Thanks, that did it!