Need some help installing pfsense or opnsense on HP T620 Plus [RESOLVED]

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
UPDATE: finally got pfsense to boot. I had to choose the ZFS/root option, and that made the USB drive a UEFI bootable device and the system booted up fine. Choosing UFS resulted in a "legacy" bootable device, but it would not boot at all. @nezach was able to boot with UFS though, but I can't explain why.

As seen in this other thread:

https://forums.servethehome.com/ind...-machines-for-network-appliance-builds.21014/

I got myself a HP T620 Plus thin client machine. Although my intent is to use Linux on it, I wanted to try out pfsense and opnsense on this machine, but so far having zero luck getting it going. I would appreciate it if anyone can give me a few hints here...

Details:
[pfsense]
- downloaded the latest 2.4.3 pfsense memstick w/ vga and made a bootable USB drive installer
- i have a 16GB Ultra Fit USB stick inside the T620+ that I plan to use as boot drive
- I boot the pfsense installer usb drive and tried to install on the internal 16gb ultra fit
====> when I booted the usb installer on any of the USB 3.0 external ports, i got erratic behavior like syslogd seg faulting and other spontaneous reboots. i then tried one of the USB 2.0 external ports and finally was able to install pfsense.
- for the install options, i just chose the guided install, GPT, and it completed, then asked me if I wanted a shell to make any manual changes, i said no. then it asked me if i wanted to reboot, so I enter yes, remove the usb installer stick, but when it tries to boot, it says no boot drive.
- i repeated the above a few times, same results. not sure what's going on...
- i then try again, but instead pick MSDOS instead of GPT. the installer warns that on this system that option isn't bootable, but neither was the GPT, so I chose to try it anyway. unfortunately, same result upon reboot says no boot drive.

[opnsense]
- i had the same problems as pfsense when booting the installer usb on a USB 3.0 port. although, this was really bad as it would spontaneously reboot several times or it would show I/O issues itself.
- I then tried the USB 2.0 port, and that was better, but not great. At least it got me to the login prompt, where I login as "installer" and it launches the installer program. but from this point forward, it was all downhill. at random points during the installer program, it would just crash and kick me back out to the login prompt. i tried several things to try to "avoid" triggering a crash, but no dice. I could never get it to install.

Keep in mind, there is also a M.2 SATA SSD in this system that has HP's Thin Pro installed. I don't want to install on that M.2 SSD as I think the USB drives are just a lot easier to replace when they fail and need replacing. So far, I haven't tried installing on the M.2 SSD at all, but that's not something I want to do right now. I didn't think it would be such a problem installing pfsense or opnsense on a USB drive. everything just seems very unstable.

Being concerned that maybe something was wrong with the hardware, I did a clean install of CentOS 7 from a usb installer and it completed smoothly and booted up just fine. I then moved the boot drive to different ports to see if maybe I just had a bad USB port, but CentOS 7 booted up quickly and without any issues on all internal USB ports and even the external USB 3.0 ports. So, at least, this gives me a little comfort that I think the hardware is fine.

Any suggestions? As for the pfsense install, since I can at least complete the install process (it just won't boot it), is there something I can do or check on at the end of the install process at the shell?
 
Last edited:

Stonent

New Member
Dec 16, 2016
18
0
1
TX
Sounds like a UEFI/Legacy issue. Some OSes will choose for you what mode and if your bios is not in that mode it won't work. See if theres an option to switch between UEFI and Legacy (also known as CSM).
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Sounds like a UEFI/Legacy issue. Some OSes will choose for you what mode and if your bios is not in that mode it won't work. See if theres an option to switch between UEFI and Legacy (also known as CSM).
yeah, i did look at that and enabled both options. in the BIOS, it shows me a boot order list. It starts with UEFI (USB CD, USB hard drive, CentOS) and then it lists Legacy (USB CD, USB HDD, SATA SSD, PXE, etc.). And it will also boot the Thin Pro SSD (M.2 16GB SSD) if the CentOS USB drive is missing.. so it looks like it will boot UEFI first (CentOS), and falls back to Legacy (when it boots Thin Pro off the SSD).

For what it's worth, CentOS is booting with UEFI as far as I can tell.

I've even enabled ALL boot options at one point just to see if something will work, but other than spending time trying to PXE boot, and ended up in the same place with no boot drive.
 

Marsh

Moderator
May 12, 2013
2,586
1,432
113
Could you do a basic install of PFsense to the USB drive using another machine?
Then, move the USB drive to the HP T620.

I sort of remember that I had to nstall on another machine , then run it on HP T5300.
I used to run about 8 x HP T5300 clients about 6-8 years ago.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Could you do a basic install of PFsense to the USB drive using another machine?
Then, move the USB drive to the HP T620.

I sort of remember that I had to nstall on another machine , then run it on HP T5300.
I used to run about 8 x HP T5300 clients about 6-8 years ago.
i guess i could do something like that in a VM? and just pass the USB drive to the VM to install on it....
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,520
238
63
Whenever pfSense does this I'd just swap to legacy and it'll work.
 

nezach

Active Member
Oct 14, 2012
210
119
43
Quick update. I think there are several different issues we are dealing with here.

Booting from 3.0 drive on 3.0 port is a no-go, but setting 'kern.cam.boot_delay="10000"' appears to take care of the problem and then I am able to boot.

Installing to 3.0 drive on 3.0 port is a no-go, I believe there is some kind of controller/driver issue, need to take a better look at this. Installing to 3.0 drive on 2.0 port (I used the one on the motherboard) worked fine, and I am able to boot from that 2.0 port just fine.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Could you do a basic install of PFsense to the USB drive using another machine?
Then, move the USB drive to the HP T620.

I sort of remember that I had to nstall on another machine , then run it on HP T5300.
I used to run about 8 x HP T5300 clients about 6-8 years ago.
sh1t... i clearly don't know how to install pfsense or something... or, does pfsense not usually install well on USB drives?

i just fired up a VM, downloaded the pfsense installer ISO, booted that ISO images and attached the USB drive.. the install process went smoothly. then reboot, with the ISO detached, and it does recognize a bootable hard drive, but just says "booting hard drive..." and then stalls forever.

Now, interestingly, although the USB drive is recognized as bootable in the VM, in the T620+ (even though I couldn't boot it in VM, I plugged it into the T620+ anyway) it won't even recognize it as bootable, even if I manually choose the USB drive from the boot options menu, it just says no boot drive. So, definitely a BIOS + pfsense bootstrap issue.

there must be something about the way pfsense boots that the BIOS/UEFI on the T620+ doesn't agree with.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Quick update. I think there are several different issues we are dealing with here.

Booting from 3.0 drive on 3.0 port is a no-go, but setting 'kern.cam.boot_delay="10000"' appears to take care of the problem and then I am able to boot.

Installing to 3.0 drive on 3.0 port is a no-go, I believe there is some kind of controller/driver issue, need to take a better look at this. Installing to 3.0 drive on 2.0 port (I used the one on the motherboard) worked fine, and I am able to boot from that 2.0 port just fine.
yup, that's exactly the same issues i ran into. basically, the USB 3.0 ports are to be avoided if using pfsense. CentOS installer worked okay, so you are probably right in that there might be a driver issue in pfsense because with Linux kernel, it seems to work fine.
 

nezach

Active Member
Oct 14, 2012
210
119
43
Since it is working for me there is no reason why it shouldn't work for you :) When installing I chose ZFS file system and did not make any other changes. I just plugged in USB stick that was booting fine from 2.0 port to 3.0 port and system booted fine.

Couple ideas:
  • try different USB stick
  • make sure secure boot is not enabled
  • load BIOS defaults and start from there
My relevant BIOS settings:

Code:
 - Storage:
   - Storage Options:
     - SATA Emulation                   AHCI
     - USB Storage Boot                 Before SATA

 - Security:
   - System Security:
     - Data Execution Prevention        Enable
     - Virtualization Technology        Disable
   - Device Security:
     - Network Controller               Device available
     - SATA0                            Device available
     - SATA1                            Device available
     - Pcie Device in Mini-PCIE Slot    Enable
   - USB Security:
     - USB Port Configuration           Auto
     - Front USB Ports                  Enable
     - USB Port 3                       Enable
     - USB Port 4                       Enable
     - USB Port 8                       Enable
     - USB Port 9                       Enable
     - Rear USB Ports                   Enable
     - USB Port 0                       Enable
     - USB Port 1                       Enable
     - Internal USB Ports               Enable
     - USB Port 2                       Enable
     - USB Port 7                       Enable
   - Slot Security:
     - PCI Express x4 Slot              Enable
     - Mini Card Slot                   Enable
   - Secure Boot Configuration:
     - Legacy Support                   Enable
     - Secure Boot                      Disable
     - Fast Boot                        Disable

 - Power:
   - Hardware Power Management:
     - SATA Power Management            Enable
     - S5 Maximum Power Savings         Disable
     - S4/S5 Wake on LAN                Disable

 - Advanced:
   - Onboard Devices:
     - Serial Port A                    IO=3F8h; IRQ=4
     - Serial Port B                    IO=2F8h; IRQ=3
     - Parallel Port                    IO=378h; IRQ=7; DMA=3
   - Bus Options:
     - PCI SERR# Generation             Enable
     - PCI VGA Palette Snooping         Disable
   - Device Options:
     - Integrated Graphics              Auto
     - Multi-Processor                  Enable
     - Internal Speaker                 Enable
     - NIC PXE Option ROM Downlaod      Disable
 
  • Like
Reactions: BLinux

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Since it is working for me there is no reason why it shouldn't work for you :) When installing I chose ZFS file system and did not make any other changes. I just plugged in USB stick that was booting fine from 2.0 port to 3.0 port and system booted fine.

Couple ideas:
  • try different USB stick
  • make sure secure boot is not enabled
  • load BIOS defaults and start from there
My relevant BIOS settings:

Code:
 - Storage:
   - Storage Options:
     - SATA Emulation                   AHCI
     - USB Storage Boot                 Before SATA

 - Security:
   - System Security:
     - Data Execution Prevention        Enable
     - Virtualization Technology        Disable
   - Device Security:
     - Network Controller               Device available
     - SATA0                            Device available
     - SATA1                            Device available
     - Pcie Device in Mini-PCIE Slot    Enable
   - USB Security:
     - USB Port Configuration           Auto
     - Front USB Ports                  Enable
     - USB Port 3                       Enable
     - USB Port 4                       Enable
     - USB Port 8                       Enable
     - USB Port 9                       Enable
     - Rear USB Ports                   Enable
     - USB Port 0                       Enable
     - USB Port 1                       Enable
     - Internal USB Ports               Enable
     - USB Port 2                       Enable
     - USB Port 7                       Enable
   - Slot Security:
     - PCI Express x4 Slot              Enable
     - Mini Card Slot                   Enable
   - Secure Boot Configuration:
     - Legacy Support                   Enable
     - Secure Boot                      Disable
     - Fast Boot                        Disable

 - Power:
   - Hardware Power Management:
     - SATA Power Management            Enable
     - S5 Maximum Power Savings         Disable
     - S4/S5 Wake on LAN                Disable

 - Advanced:
   - Onboard Devices:
     - Serial Port A                    IO=3F8h; IRQ=4
     - Serial Port B                    IO=2F8h; IRQ=3
     - Parallel Port                    IO=378h; IRQ=7; DMA=3
   - Bus Options:
     - PCI SERR# Generation             Enable
     - PCI VGA Palette Snooping         Disable
   - Device Options:
     - Integrated Graphics              Auto
     - Multi-Processor                  Enable
     - Internal Speaker                 Enable
     - NIC PXE Option ROM Downlaod      Disable
thanks. that's encouraging. i never chose the ZFS option though during install... I will check my BIOS settings to compare with above.

BTW, what is your boot order settings?
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,618
2,814
113
32
fohdeesha.com
Have you posted on the pfsense forum? They'll have more detailed info most likely considering the devs are there, they've helped me past some very strange installs. Could even be a bug with this hardware combo they'd like to know about
 

nezach

Active Member
Oct 14, 2012
210
119
43
Code:
   - Boot Order:
     - UEFI Boot Sources
       - USB Floppy/CD
       - USB Hard Drive
       - UEFI: SanDisk SDSA6MM-...        <-- SSD
     - Legacy Boot Sources
       - USB Floppy/CD
       - Hard Drive
         - SATA1
         - SanDisk                       <-- USB Stick
 

nezach

Active Member
Oct 14, 2012
210
119
43
Just installed using UFS instead of ZFS and it is working fine, so I think you are dealing with USB stick or BIOS config issue here.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
Just installed using UFS instead of ZFS and it is working fine, so I think you are dealing with USB stick or BIOS config issue here.
I've tried a few SanDisk branded USB sticks so far and having similar issues. So, I think it is probably BIOS setting issue perhaps? Did you tweak any particular setting? If not, I might just reset BIOS settings to factory defaults and see if that works. Strange thing is that it boots CentOS and Thin Pro without any issues... just having problems with pfsense boot.

BTW, can you try installing opnsense? I couldn't even get the installer to succeed with that...
 

nezach

Active Member
Oct 14, 2012
210
119
43
I flashed latest BIOS as per your instructions in the other thread (thanks for that BTW) and loaded defaults afterwards. I only changed couple minor settings like POST delay.

CentOS and ThinPro are Linux based and pf/opnsense are FreeBSD so maybe the issue lies with FreeBSD compatibility.
 

nezach

Active Member
Oct 14, 2012
210
119
43
Tried installing opnsense from USB 2.0 port. First couple times it suddenly rebooted while booting. It booted ok on third attempt, I then tried installing it to an USB stick connected to 2.0 port, and it installed and booted fine.

Try running memtest on your machine, maybe memory stick is not doing well.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,610
1,057
113
artofserver.com
I flashed latest BIOS as per your instructions in the other thread (thanks for that BTW) and loaded defaults afterwards. I only changed couple minor settings like POST delay.

CentOS and ThinPro are Linux based and pf/opnsense are FreeBSD so maybe the issue lies with FreeBSD compatibility.
Tried installing opnsense from USB 2.0 port. First couple times it suddenly rebooted while booting. It booted ok on third attempt, I then tried installing it to an USB stick connected to 2.0 port, and it installed and booted fine.

Try running memtest on your machine, maybe memory stick is not doing well.
Ok. thanks for doing that. Now, I'm just feeling really lame... no idea why I still can't boot pfsense or why opnsense installer blows up on me. going to try loading BIOS defaults later on... but right now have more urgent issues to take care of.