Dell VEP/VMWare Edge/Velo Cloud SD-WAN/VeraCloud VEP1400/VEP1400-X firewall units

[j_h_o]

From this Deal thread: https://forums.servethehome.com/ind...al-core-intel-6x1gbe-2x1gbsfp-100-ebay.39176/

This is a family of devices with Atom C3000 CPUs.

VEP1400/Edge 610: Atom C3338, 6xRJ45 1Gbps ports and 2x1Gbps SFP ports and eMMC storage
VEP1400-X units have faster CPU, more RAM, more expansion options, and 10Gbps SFP+ ports.

Has anyone gotten an Edge 610 running pfSense/OPNSense?

1. How did you update the BIOS/firmware? (How did you deal with the watchdog?)
2. How did you get the RJ45 and SFP[+] ports working?

 

[eloich]

Hello all,

I've just put my hands on an Edge 620 (apparently similar to VEP1425, in the VEP1400-X family, with the two SFP+ ports).

The unit I received had been totally wiped out so, as you may rightly guess, it rebooted every 5 minutes (watchdog kicking in).

Note that only loading Dell's DiagOS will not prevent the watchdog from activating, and there is no BIOS setting that I've been able to find that would allow the watchdog to be deactivated.

I researched the subject a bit and found out that if the regular VEP1400(x) BIOS was loaded, the watchdog could be deactivated (in fact, after loading the regular VEP1400 BIOS, even without touching any settings, the watchdog did not activate anymore).

In order to do anything you need to connect to the serial console of the unit. The serial console is located under a small plate at the rear of the unit. No need for a special cable; just a regular USB_A to micro USB cable will do. Settings are 115200 bauds, 8N1 , xon/xoff ,istrip.I'm using screen under linux and my command was sudo screen /dev/ttyUSB0 115200,cs8,-ixon,-ixoff,istrip. Here is a picture of the console port:



Here is a list of the files I used on my unit:

• DiagOS : Dell Support DiagOS . Note: DiagOS needs to be installed in order to load the BIOS.
• BIOS : zipped bios files

To install DiagOS, you only need to burn the image on a USB stick then boot from it from the EFI loader.

To install the BIOS file, you need to boot into DiagOS, copy the .bin file directly into the /root directory, then run (from the /root directory):

./VEP1400-X-BIOS-3.48.0.9-19.bin

The BIOS flash procedure initially performs a reboot of the device, then you need to re-run the command the above command from the /root directory. This will flash the BIOS, the CPLD and the PIC, in this particular order if I remember well (sorry, did not take screenshots and going from my old man's memory...)

Note: in my particular case, the CPLD flash initially failed, but after performing a cold reset and re-doing the whole flash procedure above it worked out properly.

Also note that after the BIOS flash, the boot sequence messages are much more verbose.

I've since installed PFSense (could have installed OPNSense as well, but the thing is this was the first thing that came into my hands that did not involve installation using VGA) and here are my initial findings.

First, no issues during installation on the 128Gb nvme internal drive. Which is good news I guess.

Second, FreeBSD only seems to detect both SFP+ interfaces (I've installed some cheap 10GTek rj45 modules just to test and it works great). All other 6 interfaces show up as nonen@pci00:... (where n is the interface number: so none0@pci, none1@pci, etc) when typing pciconf -lv. The NIC LEDs don't even try to light up.

Also, going into the BIOS and selecting one of those intefaces, it never detects a link (and no NIC LEDs either):

      Aptio Setup Utility - Copyright (C) 2022 American Megatrends, Inc.       
          Advanced                                                             
/----------------------------------------------------+-------------------------\
|> NIC Configuration                                 |Click to configure the   |
|                                                    |network device port.     |
|  Blink LEDs              0                         |                         |
|                                                    |                         |
|  UEFI Driver             Intel(R) 10GbE Driver     |                         |
|                          7.0.19 x64                |                         |
|  Adapter PBA             000600-000                |                         |
|  Device Name             Intel(R) Ethernet         |                         |
|                          Connection X553 1GbE      |                         |
|  Chip Type               Intel X550                |-------------------------|
|  PCI Device ID           15E5                      |><: Select Screen        |
|  PCI Address             07:00:00                  |^v: Select Item          |
|                                                    |Enter: Select            |
|  Link Status             [Disconnected]            |+/-: Change Opt.         |
|                                                    |F1: General Help         |
|  MAC Address             18:5A:58:BC:2E:E6         |F2: Previous Values      |
|  Virtual MAC Address     00:00:00:00:00:00         |F3: Optimized Defaults   |
|                                                    |F4: Save & Exit          |
|                                                    |ESC: Exit                |
\----------------------------------------------------+-------------------------/
        Version 2.19.1266. Copyright (C) 2022 American Megatrends, Inc.

Note, though, that if/when you load DiagOS, and if you connect one of the six 1Gb ports to the network, the NIC port LEDs will start blinking.

This is where I'm at right now and I'm trying to figure out whether the proper driver is getting loaded (for the Intel X553 interfaces) and/or if there is some firmware that needs to be dynamically loaded during the OS boot in order to get the NICs to even respond. I'm thinking this might be the case since loading DiagOS seems to allow the NICs to work properly.

If anyone has an idea that could help me/us going further, that would be great.

Right now, the device could work as a 2 port firewall but hey, would be great to revive the 6 other ports.

Cheers,

 

[oneplane]

I still haven't received my non-X VEP1400 platform (Edge 610), but on both the X and non-X, Dell has put a CPLD in charge of the X500 series NIC and LEDs. On the X the CPLD defaults into all-active mode but apparently on the non-X it defaults on offline mode. In Diag OS Dell added an RC script that writes to a few I2C addresses and one is commented as saying "power up SFP". Maybe running that on BSD turns the ports on without any further changes.

 

[eloich]

Thanks for your reply.

I'll try to boot into DiagOS and see if I can figure this out. Hopefully it will provide some answers.

Cheers,

 

[Ralph_IT]

Thanks everyone for sharing your findings. Got some time to tinker with my 680 and, since I got 2, the first will be my guinea pig and then I'll document all the process with the second.
My findings so far:
Connected to it with a simple usb to micro usb cable. CP210x drivers seem mandatory for putty to see anything.

Next, I wanted to install DIAG-OS in the eMMC but the process was cumbersome: I burn the image with Rufus, but it cannot start in GPT/UEFI mode, so I burnt it again with MBR.
Then the installler wiped the eMMC correctly but GRUB cannot install in MBR mode.
Back to Rufus and GPT/UEFI mode again. This time all worked and DIAG-OS was installed. Login as user root - password: calvin.

Copied the firmware updater in a USB, copy it to the eMMC and execute it as per instructions on the PDF.
It warns you about doing some reboots because it needs to change bios modes before flashing the BIOS and the CIP. So you need to reboot and execute the updater at least 3 times.

I tried to install unsuccessfully: Promox 7.3.1, PfSense CE 2.6.0, OpnSense 23.1, Untangle 16.6.2.
The USB boots, but I think I must change some options on the iso to be able to boot from console, as all I see is a grub welcome message and the prompt gets stuck.

Maybe I am too tired right now to notice what I am missing.
Tomorrow will do a little tear-down a post the pics.

 

[Ralph_IT]

(WARNING: Photo Wall)
Time for a little VMWare Edge 680 Teardown with pics:

Front

Right and Left side. Both are identical except for the Kensington lock:

Back. Under the plate there is a micro usb serial port:


Now for the funny part, disassembly. You will need
- Phillips driver (Number 1) --> 1 screw for the Micro USB serial plate + 8 bottom plate screws
- Torx driver (T10) --> 4 screws at the outer edge of the box. These keep the plastic cap in place.
- Hexagonal 10 mm Nut driver --> The power barrel has a nut of this size (and a washer! beware!)
- OPTIONAL (Pozidrive Number 1) --> 8 fan screws


Mind the fan connectors when removing the plate:


Bottom plate removed:

You cannot change the RAM nor the SSD easily just by taking apart this plate.

Plastic cap removed:

You can see the Wifi/Blutooth card and the CPU heatsink. The heatsink is massive and heavy and if you need to change the RAM or the SSD, although it may sound strange, you'll have plenty of space by removing it. So there it goes

Upper:

The thermal paste was not dried or craked, but it was not fresh either. I replaced it with some Artic MX-4 I had laying around.

Bottom:

Great news for expandability!
First, the board has a mounting position for m.2 2280, so upgrading to a 1TB or even 1.92TB ssd is possible.
And second, the original RAM stick is 16GB, which means theorically you can put a 32GB ECC SODIMM stick for a total of 48GB of RAM which I did just for testing.

Finally, exploded view and details of CPU, RAM, SSD, Wifi and fan serial numbers:








Final thoughts and findings:
I've a QNAP with a KSM26SED832HA stick. This is DDR4 32GB SODIMM ECC Unbuffered running at 2666Mhz.
Since I have to install a X520-DA2 in it, I took advantage of the shutdown time and I tried to boot the 680 with the 32GB installed.
It took forever to start the BIOS, but it started, checked OK the ECC and the system showed it:





I think I'm going to replace the wifi module with an ssd and maybe upgrade the RAM to 48GB just for braggind rights.
There is no such thing as overkill, right?

P.S.: I forgot to mention it before. When you upgrade the BIOS, it sets a password for entering it again: the S/N of the appliance plus an exclamation mark at the end.
E.g.: if your serial is 1A2B3C, the password by default will be 1A2B3C!
The upgrade documentation didn't warn you about the password but at least it tells you which one is, so read carefully.

 

[eloich]

Thanks for posting the internals. Looks pretty much like the 620 I have, although I only got one fan and the original RAM is 8gb.

I tried to install unsuccessfully: Promox 7.3.1, PfSense CE 2.6.0, OpnSense 23.1, Untangle 16.6.2.
The USB boots, but I think I must change some options on the iso to be able to boot from console, as all I see is a grub welcome message and the prompt gets stuck.

And this is where I'm kind of stuck. I've been able to install the (pfSense-CE-memstick-serial-2.6.0-RELEASE-amd64 which is the image which allows you to boot from a serial console. I now have to figure out a way (and it's out there, just haven't looked hard enough) to boot another OS (likely a linux OS) from the serial console without having to fiddle too much around. But, I can assure you pfSense CE actually installs, boots and can be configured using at least one of the SFP+ interfaces (the memstick-serial version, that is).

I'll try to find some time to boot again from DiagOS and go through the RC scripts to figure out what gets loaded and allows to speak with the remaining X553 interfaces.

Stay tuned and also let me know how it goes installing either pfSense or opnSense (memstick-serial).

And almost forgot; thanks for the tip about memory stick (probably won't need that much memory on my router/firewall, but good to know) and also for sharing the password trick as I had forgotten to point it out. This is going to be a great thread.

Cheers,

 

[eloich]

Ok. Got into DiagOS and read through /etc/rc.local .

First interesting bit is as follows, and seems to be related to the PIC watchdog. Currently wondering if this could be a workaround to flashing the BIOS (from line 15):

# Disable PIC Watch Dog
i2cset -y 1 0x22 0 0 b
# Set PIC Watch Dog default count to 0 (offset 0x0=LSB; 0x1=MSB)
i2cset -y 1 0x24 0x0 0x0 b
sleep 1
i2cset -y 1 0x24 0x1 0x0 b

The other part is related to the NICs. Starting on line 31 is the interesting bit:

value=$(i2cget -y 1 0x31 0x0 | cut -c 3-4)
boardid=$(echo $value | xxd -r -p | xxd -b | cut -d' ' -f 2)
boardtype=$(echo $boardid | cut -c 5-8)
case ${boardtype} in
"1000"|"1010"|"1011")
    # For Gris platform reinit the Marvell phy and I350 MAC
    velo_ver=$(/opt/dellemc/diag/bin/nvramtool -r -R 0x59 | grep VeloCloud | awk '{ print $NF }')
    if [ -z $velo_ver ]; then
        :
    elif [ $velo_ver = "1" ]; then
        rm /etc/udev/rules.d/70-persistent-net.rules
        rmmod ixgbe
        i2cset -y 1 0x31 0x1 0xff
        sleep 1
        while [ -z "$(lsmod | grep igb)" ] && [ $count -le $timeout ]
        do
            echo 1 > /sys/bus/pci/rescan
            echo "Rescan PCI bus..."
            count=$(( count+1 ))
            sleep 1
        done
        modprobe ixgbe
    fi
    ;;
"1001")
    ;;
"1100"|"1101"|"1110"|"1111")
    ;;
*)
    echo "Unkown Board Type." > /dev/kmsg
    ;;
esac

In my case, board id is 00101000, which in return means that it's a VeloCloud version board. This would call for the actions in the 70-persistent-net-rules (rmmod ixgbe, i2cset -y 1 0x31 0x1 0xff, then rescan PCI bus with the newly found igb ports). Just need to figure out a way to shove this into nnSense / FreeBSD.

Note that I've got the rc.local file handy and will post it if it actually helps (granted that I'm allowed to to so, of course).

By the way, here is a screenshot of PfSense console after booting up:

*** Welcome to Netgate pfSense Plus 23.01-RELEASE (amd64) on pfSense ***

 WAN (wan)       -> ix0        -> v4/DHCP4: 192.168.2.186/24
 LAN (lan)       -> ix1        -> v4: 192.168.1.1/24

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell + Netgate pfSense Plus tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Enable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

Enter an option:

As well as a shot of the dashboard:


Now, hunting for a way to shove those commands into FreeBSD... Stay tuned!

 

[oneplane]

Excellent, I knew there had to be something in the DiagOS that was needed on the non-X platform. On all the X-platforms the system behaved the same regardless of the I2C commands, so I figured this must be for different models. Especially the PHY notes (those were in each DiagOS version I have seen so far) seem to indicate that some boards use a PHY for the C3000 integrated interfaces that needs to be powered up separately.

I did end up using the I2C commands for fan and brownout control in one case (an install with VyOS in a somewhat sub-standard mounting location), but since that's all Linux, the entire i2c-util package ports straight over.

As for interface configuration, on OpnSense I generally have excellent performance on non-IDS configurations with hardware offloading on and VLAN offloading on, most of the configurations use DAC cables between the VEP and switches, and have 1 or 2 GBE connections for out-of-band management and one LTE fallback (because ISPs often want to ship you their "LTE modem" over here... 90's mentality). Works great, can do 10G bi-directional on both interfaces without breaking a sweat. So as soon as you get the NICs to cooperate, they will work in practically any configuration. And that leads me to believe that the PIC and the CPLD are not so much involved in the data or control paths, but mostly just power-on and diagnostics (including port LEDs on all ports except the non-X500 ports). The front LED seems to be controlled purely by the PIC, and not by the CPLD as far as I can tell.

In addition, here is the HWProbe log for a setup in a slightly toasty industrial environment (the air is filtered, and the power is UPS'ed, but the temperature is... not amazing): HW probe of Dell EMC VEP1425-V210-CPU A00 Desktop Computer (VEP1425-V210) #6faf4aed53 this is generated using the HWProbe package in OpnSense and shows the devices, ID's, drivers etc.

 

[eloich]

Hmmm, interesting. If I understand correctly, there are 4 10gb ports; 2 of which are SFP+, and two more are rj45, right?

Do you happen to know which of the labeled GEn (GE1 .. GE6) are the 10gb ports? I seem to remember that the initial VeloCloud setup used GE5 and GE6, but for the life of me I cannot find the info anymore. Also, none of these ports show up in BSD... (well, they do show up but no driver gets loaded).

Thanks; seems like we're still going forward on this.

 

[oneplane]

So there are four ix* ports, ix0 and ix1 are the SFPs, ix2 and ix3 are Copper. I do suspect that they are 10G ports, but the (Marvell?) PHY they used are GbE only.
All of the ix* ports are SR-IOV capable, so that's fun.

I have checked our the I2C access on FreeBSD but I don't seem to have device nodes for them. The buses do show up:

ichsmb0: <Intel Denverton SMBus controller> port 0xe000-0xe01f mem 0xdff56000-0xdff560ff irq 23 at device 31.4 on pci0
smbus0: <System Management Bus> on ichsmb0
ismt0: <Atom Processor C3000 SMBus 2.0> mem 0xdff5c000-0xdff5c3ff irq 16 at device 18.0 on pci0
smbus1: <System Management Bus> on ismt0

But in /dev there is no iic/smb/ismt... I suspect the kernels get built with drivers but no bus support.

There are built-in i2c and smbus commands in OpnSense so we're 66% there, just the bus nodes for dev missing...

Edit: a little bit of rooting around; it looks like devd/devfs is secured by default and hides all non-essential devices, which is good. But IIC/I2C is never unhidden it seems so that would explain the missing device nodes.

Edit2: I guess that's not it either, loading the ismt driver works, but it doesn't create any iic0/iic1 devices for the I2C bus...

 

[eloich]

Well, I've got good news and bad news...

First, good news: I've got all NICs working. PfSense is now seeing the all the NICs in all their glory, and they can even be selected in the GUI interfaces menu. See below:

ix1@pci0:5:0:1: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15c4 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 10 GbE SFP+'
    class      = network
    subclass   = ethernet
ix2@pci0:7:0:0: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15e5 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 1GbE'
    class      = network
    subclass   = ethernet
ix3@pci0:7:0:1: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15e5 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 1GbE'
    class      = network
    subclass   = ethernet

And now, the bad news.

I really hate to say it, but I got excited and fell into my bad habits of doing more than one thing at a time before retesting, and as such I don't know what fixed it exactly.

Here is what I did anyways, as it may help somebody else in the future.

Went on to figure which of the GE labeled ports were 1gb and which were 10gb, dismantling the unit (removing the top plastic cover) and kind of stumbled on two small pusbutton switches besides the NIC ports:



Booted the unit up and kind of stumbled on the switches (didn't really want to press them but my old clumsy fingers did not agree) and connected GE5 to a cable and the LEDs lit up. Now, guess what, I can see the interfaces just fine. I'm both happy and rather frustrated at this point.

One thing of note: during that first boot-up with the buttons briefly pressed (probably both at or near same time, rather pissed at myself right now, even though it might not even be related, but I guess we might never know), the BIOS loading sequence was slower than usual and it also seems it performed a reset before booting completely normally.

Anyways, here it is; it's running totally fine. The only thing that's not figured out (aside from why it's working actually...) is the wifi portion, for which I don't really care anyways. The fan (only one fan on this unit) is a bit loud but I'll figure it out. Won't be able to hear it when it's in the closet anyways.

I'll keep testing and report as I go. Let me know if I can be of any help to anyone.

On a separate note, there are a couple of jumpers on that board and leads. These got me curious, but unless there is something I feel I'm really missing, I'm not really willing to do anything at this point.

Cheers,

 

[oneplane]

The board has a ton of jumpers and headers on the edges, it's pretty wild. And the fact that it also has a CPLD.. AND a PIC microcontroller.. AND a SuperIO that also has an embedded controller??? It's like they built this thing with 4-computers-in-1!

I suspect the buttons are for selecting a preset of sorts (either in the PIC or in the CPLD) and it is persistent. I would love to get some board schematics for this thing. It almost seems like this is a NUC on steroids combined with a Raspberry Pi.

 

[eloich]

Just one more thing before I head over to other house chores...

Figured out which ports were actually 10gb; they're the ones labeled as Ge5 and Ge6, so both rj45 ports at the very right, near the power port.



Finally, and that's probably just for my unit, the fan is rather noisy and upon closer inspection, seems like the unit must have been operated in a rather dusty environment in the past. Removed the fan, peeled the label over the bearing and applied a drop of light oil, which really helped (probably half less noise, which should help with the WAF in the near future, if I decide to deploy the unit as my main firewall; contemplating 1.5gb service over here, but I'm on my guards since the provider is Bell and I've had nothing but issues in the past -- that's 15 years ago though). Ordered a 40x10 12V 3 pin fan and I'll swap it in when it arrives.

Cheers,

 

[oneplane]

Just one more thing before I head over to other house chores...

Figured out which ports were actually 10gb; they're the ones labeled as Ge5 and Ge6, so both rj45 ports at the very right, near the power port.

View attachment 27836

Finally, and that's probably just for my unit, the fan is rather noisy and upon closer inspection, seems like the unit must have been operated in a rather dusty environment in the past. Removed the fan, peeled the label over the bearing and applied a drop of light oil, which really helped (probably half less noise, which should help with the WAF in the near future, if I decide to deploy the unit as my main firewall; contemplating 1.5gb service over here, but I'm on my guards since the provider is Bell and I've had nothing but issues in the past -- that's 15 years ago though). Ordered a 40x10 12V 3 pin fan and I'll swap it in when it arrives.

Cheers,

Nice! This probably also means they are not ix2 and ix3? and either those are simply IOV duplicates of the SFP ports, or it means that the buttons also switched the port modes.

 

[eloich]

From what I can see, they are indeed ix2 and ix3, as can be seen in what pciconf -lv returns above. PfSense also shows a total of 8 interfaces; 4 ix interfaces and 4 ig interfaces. Here is a bigger snippet of what the command returns (you can actually see the Atheros wireless device in there, showing no driver is actually loaded):

igb2@pci0:2:0:2:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0xffff subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb3@pci0:2:0:3:        class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0xffff subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
none5@pci0:4:0:0:       class=0x028000 rev=0x00 hdr=0x00 vendor=0x168c device=0x003c subvendor=0x0000 subdevice=0x0000
    vendor     = 'Qualcomm Atheros'
    device     = 'QCA986x/988x 802.11ac Wireless Network Adapter'
    class      = network
ix0@pci0:5:0:0: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15c4 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 10 GbE SFP+'
    class      = network
    subclass   = ethernet
ix1@pci0:5:0:1: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15c4 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 10 GbE SFP+'
    class      = network
    subclass   = ethernet
ix2@pci0:7:0:0: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15e5 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 1GbE'
    class      = network
    subclass   = ethernet
ix3@pci0:7:0:1: class=0x020000 rev=0x11 hdr=0x00 vendor=0x8086 device=0x15e5 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X553 1GbE'
    class      = network
    subclass   = ethernet

And here is a more readable version from pfSense's txt console:

 WAN (wan)       -> ix0        -> v4/DHCP4: 192.168.2.186/24
 LAN (lan)       -> ix1        -> v4: 192.168.1.1/24
 GE3 (opt1)      -> igb0       ->
 GE4 (opt2)      -> igb1       ->
 GE1 (opt3)      -> igb2       ->
 GE2 (opt4)      -> igb3       ->
 GE6 (opt5)      -> ix2        ->
 GE5 (opt6)      -> ix3        ->

Therefore, these are pretty capable, feature rich units.

 

[Ralph_IT]

I have to admit that all this rc.local and i2c and smbus commands are waaay out of my knowledge.
Will have a look at them and try to understand what they do.

Now I, too, have good and bad news.
Good news is I managed to install Untangle. Just burnt the ISO and modified the /isolinux/txt.cfg and the /boot/grub/grub.cfg files, adding

console=ttyS0,115200

just after

[...]ngfw,untangle

Bad news is Grub failed miserably to install itself on any disk, leaving a complete installation that I cannot boot from.
The problem with grub was that it was searching for any other OS installed, found the DIAG-OS installation and wanted to install itself on the eMMC, which seems impossible.
After this, weird things begin to happen on the unit:
- Fans are much much louder than before. Temps shown by BIOS are normal and even on the cool side, but fans spins at >4000 rpm.
- I need to move within BIOS menus by pressing and holding the [CTRL] key. Otherwise ANY key I press is read as [ESC].

Will give Untangle another try and if it fails again, send it to the end of the queue and try others OSes.

And here is a more readable version from pfSense's txt console:

 WAN (wan)       -> ix0        -> v4/DHCP4: 192.168.2.186/24
LAN (lan)       -> ix1        -> v4: 192.168.1.1/24
GE3 (opt1)      -> igb0       ->
GE4 (opt2)      -> igb1       ->
GE1 (opt3)      -> igb2       ->
GE2 (opt4)      -> igb3       ->
GE6 (opt5)      -> ix2        ->
GE5 (opt6)      -> ix3        ->

Therefore, these are pretty capable, feature rich units.

I am too dense right now. Does that mean that all units are capable of 10Gbe on 4 ports? 2 x SFP+ and 2 x Cooper?

 

[oneplane]

AFAICT, ix2 and ix3 are still limited to 1G by the PHY: 'Ethernet Connection X553 1GbE'

 

[heromode]

Following this interesting thread i noticed there are also rack mounts that holds 2 of these: VeloCloud 610 Rackhalterung | eBay

 

[Ralph_IT]

AFAICT, ix2 and ix3 are still limited to 1G by the PHY: 'Ethernet Connection X553 1GbE'

Ok. Just got carried by the hype.

Following this interesting thread i noticed there are also rack mounts that holds 2 of these: VeloCloud 610 Rackhalterung | eBay

I saw this auction before: 100€ for a rackmount that's 30$ also new. It is really insane.

Good news: OPNSense installs without problems on my 680. I was surprised by how smooth the installation was. All interfaces get detected and if I connect any wire to the switch, a message in the console appeared turning up this interface. So kudos for the OPNSense team.


I am not testing in deep, right now I just want to know which distros are compatible with this hardware without having to fiddle too much with configs and what not.