OMV on Pi4/8gb speed and load?

[Greg_E]

So I'm finally getting a grasp on OMV and running it on a Pi4. Target goal is SMB and NFS shares to use with my Kubernetes lab, I'm not sure I have enough resources for Longhorn on my little cluster. Is this thing really going to be fast enough to function?

Pi4/8gb housed in an Argon One with the m.2 SATA base and a 1tb drive installed, this is the data drive for OMV shares. This attaches the m.2 over USB3.0, best I can do right now. This is going to be the persistent volume for containers until I try Longhorn, if Longhorn works, I'll probably keep it running or move to Truenas when I move the rack of mini-pc into the main lab.

I will say, I'm not liking the user interface on OMV, I've gone through Freenas 9 up to Scale 25.x and I guess I'm just more accustom to it. Getting the hang of it, but just feels bulky.

 

[louie1961]

My very first home lab device was a pi 4 with 8gb of RAM running OMV. I followed a video from Network Chuck on how to set it up. I ran it with two SATA SSDs plugged into USB to SATA adapters. I had a cool little frame/case for it all as well. That was my only NAS for quite a while. Worked great actually. I also used rclone running in a script to backup the volumes nightly to Amazon Glacier. I do think however that you will find using the NFS driver in K8S for persistent volumes may not be satisfactory.

 

[Greg_E]

Nothing about Kubenetes in this cluster is going to be great, all of it is on gigabit. My real hope is that I have enough cpu and ram to get Longhorn v1 running, and have enough overhead for Rancher. Setting up a few simple workloads like a LAMP stack or simple Wordpress site would be nice too. My goal is still focused on making Harvester work properly, anything left over after that is just a bonus. When both parts are combined, I may be able to use Longhorn on the Harvester cluster for all the storage, this would be much better since Harvester has 10g connections, or 25g if I can find the bottleneck. The full lab also has Truenas on a 10g connection I could use if I need NFS.

The Pi is simply a way to get it going with a low watt budget, and I had it from years before sitting and waiting. The kubernetes nodes are Intel n95 with 12gb of ram and I'm probably going to swap the drives out to a 500GB in each. I have three of these for this task, all are POE powered through some 12v/4 amp POE++ splitters. https://forums.servethehome.com/index.php?threads/dreamquest-n95-mini-pc.55230/

Eventually the Pi needs to be a home entertainment and NAS for my media, I've messed around with multiple variations and just never applied enough time to make it do what I wanted.

 

[louie1961]

I don't yet do Kubernetes on bare hardware. I am currently running a cluster of VMs in Proxmox, more or less for learning. I don't need K3S for any of my current websites or other critical services. But what I can tell you is to run as much stuff local to each machine as you can. Longhorn running on a worker node doesn't take up much in the way of resources, and you could always partition the local disk to make one partition a long horn partition. I do something similar. I added a 200gb qemu virtual disk to each VM for longhorn to own. Kubernetes is meant to be stateless. Trying to add in NFS works with the community driver, but honestly, file storage is not as good as block storage for K8S/K3S. It's just not as performant. I only use NFS as a backup source for my Longhorn disks. I don't use Harvester, so YMMV

 

[louie1961]

BTW, one thought for you. Since you are connecting your M.2 drive via USB, you gain no speed advantage from the M.2 drive. If you have larger SATA 2.5 inch SSDs, you may as well use those. Also, OMV doesn't work well if you try to create a RAID array with USB drives. You can work around their GUI (which will try to stop you from using USB drives in an array), and I have done it, but it never really worked out well.

 

[Greg_E]

The m.2 is built into the base of the case, one of those funny USB A to A jumper blocks to connect the two halves. It's a nice design, has a pwm fan and power button. Only thing they couldn't do is give you external access to the SD card, so that's a pain.

Should I use the iSCSI hack for OMV and connect the cluster over iSCSI? The entire disk is going to be dedicated to kubernetes for now, so no risk of losing any other shares.

And truthfully, as long as things don't time out, I don't really care how fast or slow anything is moving on the kubernetes cluster. that again gives you time to look at node performance and see what's happening. Slow was a feature in the old Cisco simulation tools so you could watch the packets bounce back and forth.

 

[louie1961]

I am sure a CSI driver exists for iSCSI for K8S (or in my case K3S), but I am unfamiliar with it TBH. I have also never tried to do iSCSI on OMV.

 

[Greg_E]

I've managed to break OMV twice. Trying openSUSE LEAP Micro on a pi4/2gb with the 1tb drive, challenges here, but making some progress with nfs. Need to export a share and test it. Then I need to build up the mini-pc, again leap micro, and get rke2, nfs, and maybe longhorn set up. Sounds so easy...

 

[Greg_E]

Broke OMV again on a fresh note install. When you upgrade to latest, it decides to shut off the networking. You then need local keyboard and dispatch to run sudo omv-firstaid and set up the network again. It's back up, smb seems to be working, need to try nfs still.

Makes me wish more work has been done to run Truenas on Pi, a bigger project than I can attempt right now.

 

[louie1961]

I have never had OMV break on me in any way. Its just Debian under the hood. Upgrades have always gone smoothly for me. My only real gripe with OMV is the way salt is used to manage changes. I find the click here to apply changes banner to be a nuisance. But the real issue with it is if you try do updates from the command line it get's stuck waiting for you to apply updates.

As far as TrueNAS, I don't believe TrueNAS has a ARM compatible kernel.

 

[Greg_E]

Yeah, this was the third time, found a few posts online that helped me fix it.

Truenas does have an unofficial ARM build, but I don't think anything has been done to it after the initial, proof of build. They are really aiming at the ARM servers, but it carries into Pi as well. I need to look into it more, it also seems to need Pi5 due to additional boot modes, no way I'm buying Pi5 at current prices!

If I had large nvme drives sitting around, I'd buy an rk3588 has board, or if someone builds an sata version. I have 3 more 1tb m.2 sata drives I could put into one of these.

 

[Greg_E]

Write up of Truenas on Pi5:

How to install TrueNAS on a Raspberry Pi

Now that Joel0 in the TrueNAS community has created a fork of TrueNAS that runs on Arm, I thought I'd give it a spin—on a Raspberry Pi. I currently run an Ampere Arm server in my rack with Linux and ZFS as my primary storage server, and a Raspberry Pi with four SATA SSDs and ZFS as backup...

www.jeffgeerling.com