booting pfsense on 5018A-FTN4 server?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Kaishi

Member
Mar 7, 2011
49
2
8
Introduction:
I've got a SuperMicro SuperServer 5018A-FTN4, which I bought specifically to use as a gateway for my network. I intended to use pfsense on it, and have been attempting to install pfsense2.4.5-RELEASE-p1 on a pair of SATA SSDs.

Hardware in question:
The system in question runs an Atom C2758 processor (Rangeley) and has full UEFI support. Motherboard model is: A1SRi-2758F and I have it running with 32GB of ECC DDR3. I've updated the motherboard firmware and the IPMI firmware to the latest versions.

Storage is a pair of Samsung 860 Evo drives, connected to SATA0 and SATA1, which are on the SATA gen 3 controller. Controller is configured for ACHI.

My intention was to use ZFS-on-root and install to the dual SSDs as a mirrored setup. I configured the system for full UEFI mode (minus secure boot), and my pfsense thumbdrive (with the above version image applied) booted right up in EFI mode.

Things I've already tried:
I installed the OS using GPT(UEFI) partition scheme, no errors, and then rebooted the system. Nothing. Motherboard doesn't even see that there's an OS installed.

So, I downloaded the ISO, and mounted it via IPMI, and performed another install. No difference.

I tried putting the system into full BIOS mode (CSM on, all OpRoms set to Legacy mode), and MBR(BIOS) partition scheme. Nothing.

I tried AUTO(UFS) mode, to a single disk, in MBR mode, that's as compatible as I could muster. Still can't boot.

I then downloaded FreeBSD12.1-RELEASE and attempted to install that. Once again, the ZFS install completed successfully but the OS wasn't found by the motherboard, EFI couldn't boot it.

So I switched to a Windows 10 ISO, put the system into complete UEFI mode. OS installer boots just fine, and installs just fine, boots just fine.

What I need now:
I tried searching on the STH forums and saw a whole lot of folks discussing using this specific model for pfsense. Also, NetGate themselves sold a version of the server, nearly identical to my configuration. I'm at a loss for what to do next.

EDIT: I've spent a few years working on this system without no success. I'm ready to give up.
 
Last edited:

Kaishi

Member
Mar 7, 2011
49
2
8
I've been looking at some more documentation on freeBSD and came away with the idea that it might be related to the pfsense installer not calling "efibootmgr" after install. Or maybe the board only looking for the backup boot file (bootx64.efi) rather than a configured one.
 

Kaishi

Member
Mar 7, 2011
49
2
8
1603297799087.png
This is on a fresh install, at the end of the install I tried to use the final config shell to use efibootmgr, and got the same error. So I rebooted back into the ISO and used the rescue shell. Same error.

Clearly I'm booted in UEFI mode, when loading the ISO. But I need to write the EFI variables to the firmware so that I can boot the environment.

I've read tons of documentation and tried lots of things but this is as far as I seem to be able to get on my own.
 

ske4za

Member
Feb 4, 2019
80
45
18
I'm running the same unit on a single SSD (UFS) on pfSense without issues. I'm not sure what I can do to help? Let me know if there is anything I can do. I reinstalled pfSense from scratch last year and didn't have any issues.
 

Kaishi

Member
Mar 7, 2011
49
2
8
I'm running the same unit on a single SSD (UFS) on pfSense without issues. I'm not sure what I can do to help? Let me know if there is anything I can do. I reinstalled pfSense from scratch last year and didn't have any issues.
Is your install in native UEFI mode? Or are you using any BIOS-compatibility options? For example, the Compatibility Support Module (CSM)
 

ske4za

Member
Feb 4, 2019
80
45
18
Is your install in native UEFI mode? Or are you using any BIOS-compatibility options? For example, the Compatibility Support Module (CSM)
Can't confirm that from IPMI so I'll have to reboot at some point to see. From what I can tell on the CLI I'm not booting UEFI, so I must be using CSM/Legacy etc.
 

Kaishi

Member
Mar 7, 2011
49
2
8
I tried installing TrueNAS-12.0-RELEASE and had the exact same experience. Nothing shows up in the EFI for me to boot.

EDIT: I'm starting to think this board's firmware is hardcoded to look specifically for Windows boot EFI files and nothing else.

EDIT2: I decided to try installing VyOS, because it includes grub and grub has some workarounds for these kinds of things. Also VyOS has full UEFI support as of version 1.2

Guess what? Same result. No boot entries written, nothing to boot.

1603477875001.png

EDIT3: I'm actually installing Hyper-V server now, as a workaround for this junk. If that can boot in UEFI mode, then fine. I'll just make a pfsense VM and be done with it. Ugh.
 
Last edited:

Kaishi

Member
Mar 7, 2011
49
2
8
I called SuperMicro to ask them about it. I sent them an email linking to this thread but also including the serial number, MAC addresses, etc.

Also, I managed to reach another owner via Reddit, one who is using this exact board for his own pfsense box, and isn't having the issues that I'm having. He sent me screenshots of his whole config, which I replicated as a test. Nothing changed, system cannot boot.

I tried IDE mode for the SATA controller. Nothing.

I'm debating throwing a thumbdrive into the system and installing to that. Dunno what I would do with the SSDs afterward, though.
 

Kaishi

Member
Mar 7, 2011
49
2
8
After speaking at length with SuperMicro, they indicated that they would like me to RMA this board. I'm not sure if this is a symptom of the C2000 bug but, it's surely annoying.
 

Kaishi

Member
Mar 7, 2011
49
2
8
Replacement board arrived, I'll be installing it today, and seeing if that fixes the issue.

EDIT: so far, not much better. I haven't had it boot pfsense. Now I'm starting to wonder about my SSDs being somehow incompatible with this board, or needing some kind of firmware update. They're Samsung 860 Evo though, which is a good SATA SSD, that's my understanding.
 
Last edited:

ske4za

Member
Feb 4, 2019
80
45
18
I tried AUTO(UFS) mode, to a single disk, in MBR mode, that's as compatible as I could muster. Still can't boot.
Have you tried this with the replacement board? I'm not sure why I haven't had any trouble with mine, and I've installed pfsense twice on it now (original SSD died).
 

Kaishi

Member
Mar 7, 2011
49
2
8
I haven't tried it yet. I was disheartened when I couldn't get it to boot in UEFI mode but with the CSM enabled.

I've confirmed that the system totally loves Windows 10 and has no problems at all running. I even got all the drivers working.

I have 2 ideas about what could be the issue:

1) it's a problem with how pfsense / freebsd writes their GPT / EFI partitions, which is actually documented in the UEFI section of the FreeBSD Wiki. I think it could be this because of the board not showing that the SSDs are connected once they've been formatted by pfsense's installer, and how they don't come back to showing in the firmware until I use diskpart to clean them, or install Windows.

2) it's a hardware compatibility issue with these SSDs and motherboard. So I started looking on SuperMicro's QVL for SSDs that I could buy to replace the (brand new) Samsung 860 Evo disks I've installed.
 
Last edited:

ske4za

Member
Feb 4, 2019
80
45
18
I checked, and I think mine is a UEFI install:

Code:
gpart show
=>       40  488397088  ada0  GPT  (233G)
         40     409600     1  efi  (200M)
     409640  479838208     2  freebsd-ufs  (229G)
  480247848    8149279     3  freebsd-swap  (3.9G)
  488397127          1        - free -  (512B)
And my SSD info:

Code:
geom disk list
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r2w2e5
   descr: WDC WDS250G2B0A-00SM50
   lunid: 5001b448b81c5a1e
   ident: 191851805828
   rotationrate: 0
   fwsectors: 63
   fwheads: 16
Not sure if that helps at all :confused:
 

Kaishi

Member
Mar 7, 2011
49
2
8
I checked, and I think mine is a UEFI install:

Code:
gpart show
=>       40  488397088  ada0  GPT  (233G)
         40     409600     1  efi  (200M)
     409640  479838208     2  freebsd-ufs  (229G)
  480247848    8149279     3  freebsd-swap  (3.9G)
  488397127          1        - free -  (512B)
And my SSD info:

Code:
geom disk list
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r2w2e5
   descr: WDC WDS250G2B0A-00SM50
   lunid: 5001b448b81c5a1e
   ident: 191851805828
   rotationrate: 0
   fwsectors: 63
   fwheads: 16
Not sure if that helps at all :confused:
That does help, thanks for the information. At this point, I'm considering throwing down for either a Netgate subscription, or, for a C3000 board.
 

Kaishi

Member
Mar 7, 2011
49
2
8
@ske4za would you be able to post screenshots of your BIOS settings? I want to try a 1:1 test to see if it is something with my board or my SSDs.
 

ske4za

Member
Feb 4, 2019
80
45
18
Lucky for you, I came home from work today and the UPS connected to my pfSense decided to just shut down while I was at work. So here are some pics. I will say that it seems to bounce between the Supermicro POST graphic and the Intel Boot Agent ROM about 3 times, so I'm not entirely sure that it is actually booting in UEFI. To be honest I hadn't really given this much thought as I just installed from the thumb drive, reboot, and it's loaded. If I have time this weekend I'll turn CSM support off and see if it still boots correctly.
 

Attachments

Kaishi

Member
Mar 7, 2011
49
2
8
Lucky for you, I came home from work today and the UPS connected to my pfSense decided to just shut down while I was at work. So here are some pics. I will say that it seems to bounce between the Supermicro POST graphic and the Intel Boot Agent ROM about 3 times, so I'm not entirely sure that it is actually booting in UEFI. To be honest I hadn't really given this much thought as I just installed from the thumb drive, reboot, and it's loaded. If I have time this weekend I'll turn CSM support off and see if it still boots correctly.
So, I broke down and replaced the SSDs. I picked up some cheap PNY 120GB SATA3 ones, and installed them.

Right out of the box, the motherboard's firmware doesn't see the disks. SATA controller says nothing is attached. I enable the CSM, for good measure. I make sure Secureboot is turned off. I connect my USB3 pfsense installer.

I boot the pfsense installer with no issues. Installer sees the disks. I use the guided setup and configure a ZFS mirror on the two disks, and MBR partition scheme for BIOS, but otherwise leave all the defaults. Install completes, I remove USB, and reboot.

No change. System does not see the OS, does not see the SSDs connected to the SATA ports. It doesn't show any "boot from AHCI disk" option. System does not want to boot anything MBR best as I can see.

I will try one more time with installing in UEFI mode. but in my experience thusfar, I don't expect anything to be different.

My last possible attempt is to use an HDD or SATADOM instead of 2x SSDs.

It's so strange to see every OS installer able to recognize the disks, see the existing partitions, but not be able to write a bootable partition in a way that the board will recognize. This motherboard's UEFI firmware is from 2019. It isn't that old.

Why does it boot UEFI for Windows but nothing else?

I've spent so much time and money. I'm ready to wash my hands of the whole thing.
 

Kaishi

Member
Mar 7, 2011
49
2
8
I found a very interesting post from 2014 on the netgate forums:

Installing pfSense on a Supermicro 5018A-FTN4 SuperServer | Netgate Forum

Suffice to say, someone indicated that this board is picky about SSD controllers. They cited that it didn't like Marvell controllers in some SSDs they tried.

While neither of the SSD models I've tried have used Marvell controllers, I am wagering that the board is just picky about SSDs in general. So, I've ordered some used Intel SSDs that are on the QVL for this specific motherboard.

If that doesn't resolve the issue, I don't know what will.