SmartOS / Danube Cloud on modern-ish systems

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Parallax

Active Member
Nov 8, 2020
417
208
43
London, UK
Hello,

In my quest to try to find a combined container/VM management platform for my homelab, my journey has taken me to SmartOS, Triton Datacentre, and eventually Danube Cloud as a nice way to have the best of LXCs, Docker, Bhyve and KVM all in one lean platform, with a robust implementation of ZFS to boot (literally) and a GUI to manage it all.

The trouble is, I'm admitting defeat on actually booting it. I'm running all Lenovo P350 Tinys (so, 11th Gen systems), and either SmartOS itself or Danube Cloud will both boot off a USB, show a menu, then unpack the compressed system, then I get "Booting ..." on a blue background and then it just hangs forever. OmniOS clears the screen after the booting message but then nothing.

I've read everything I can find to try to resolve, since I have no extra output to go on even with verbose set. I have tried both physical USB keys and attaching images via PiKVM with no change in behaviour. The only thing I can see which may be a concern is the fact that the P350s USB controllers are all USB 3.x. I'm also unable to get it to iPXE boot - SmartOS seems not to be on the netboot.xyz menu any more.

Does anyone have any bright ideas/experience to offer?
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
I do not use SmartOS as I need the global zone that is quite restricted there for a general use filer (may change if I find more time as SmartOS with all its VM server features, running from RAM, boot from an USB key without special dependencies is a perfect base for many server services).

If the docs Overview - SmartOS Docs or the wiki SmartOS - OpenStack are not helpful and you get no other tip here, check and ask at smartos-discuss, Topicbox
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
Just curious as to why Truenas Scale would not fit your needs. Docker and KVM are supported. Are LCX and Bhyve critical?
Truenas scale is much better supported.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
It depends.
Sometimes a more specialized BSD, OSX or Solaris Unix fits better than a mainstream Windows or one of the LInux variants.
 

Parallax

Active Member
Nov 8, 2020
417
208
43
London, UK
Just curious as to why Truenas Scale would not fit your needs. Docker and KVM are supported. Are LCX and Bhyve critical?
Truenas scale is much better supported.
It seems whenever I give an explanation about my concerns with TrueNAS Scale its hordes of supporters descend on me accusing me of heresy and no good comes of it.

In short :
- The controls for VMs are awful. No easy way to see the resources being actually consumed by each, no way to operate a cluster, fail over from node to node, etc. In short nothing of what you'd expect for VM management present at all, and there's no GitOps integration either.
- The implementation of k8s is very opinionated, different from and not interoperable with an existing cluster, again with minimal visibility, no way to build up a cluster, no way to monitor resource consumption, etc. No GitOps.
- If you just want to run Docker containers it is infinitely worse as a manager than Portainer, which does not work without open heart surgery on Scale. No GitOps (I have all my compose files in a Git, for example).

I have a whole pile of other concerns but those are the showstoppers for me.

If you want more detail, you can Google but there are many discussions on things like the shortcomings of the Docker implementation.
A year ago another guy and I got into the general concerns in detail on the TrueNAS forums until the thread was locked.

I do in fact run Scale on my NAS box, but it's just not workable in a serious lab environment to manage the virtualisation so I do not run it elsewhere.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
I will not argue with you on the specifics of the TrueNAS Scale. I agree that their GUI is highly opinionated and not as intuitive.
I used to run vSphere and docker as a VM with Portainer to manage the rest of the containers. It wasn't ideal, and I wanted to consolidate storage, VMs, and containers. I foolishly decided that a mid-range QNAP NAS is the way to go. I do a lot of things right, but the ease of management nor version quality control isn't one of them. Ok, enough about the crappy prosumer NASes.

I get the feeling that you know more than enough Linux well for both of us, and chasing after GUI for core management subsystems seems optional for you. Have you considered building your own based on a plain vanilla Linux distro? As for specific functions, GUI, Portainer for Docker/LXC, Virt Tools for KVM VMs, and Poolsman for ZFS GUI.
 

Parallax

Active Member
Nov 8, 2020
417
208
43
London, UK
I will not argue with you on the specifics of the TrueNAS Scale. I agree that their GUI is highly opinionated and not as intuitive.
I used to run vSphere and docker as a VM with Portainer to manage the rest of the containers. It wasn't ideal, and I wanted to consolidate storage, VMs, and containers. I foolishly decided that a mid-range QNAP NAS is the way to go. I do a lot of things right, but the ease of management nor version quality control isn't one of them. Ok, enough about the crappy prosumer NASes.

I get the feeling that you know more than enough Linux well for both of us, and chasing after GUI for core management subsystems seems optional for you. Have you considered building your own based on a plain vanilla Linux distro? As for specific functions, GUI, Portainer for Docker/LXC, Virt Tools for KVM VMs, and Poolsman for ZFS GUI.
Thanks for the suggestions; I'll add it into my thinking. I've also recently been mulling whether LXD + MAAS + LXD Ware might do the trick. MAAS rather constrains you to Ubuntu rather than Debian which is my go-to OS; but it looks like you could run individual Docker containers in LXCs which would be nifty in a Fargate/Firecracker sort of way as long as I don't need to run a zillion Portainer agents. I don't really need to run multiple OSes for my VMs (and it's easier not to anyway).

Also I see 45 Drives has a Cockpit-based manager for ZFS as well.
 
  • Like
Reactions: BoredSysadmin