Interested in SmartOS/FIFO - migration from ESXi experiences?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Avery Freeman

New Member
Aug 25, 2016
3
0
1
42
Hi,

I have a small ESXi server running on a supermicro X9SCL-F build with a Xeon 1230v2.

I have 6 2TB drives using PCI passthrough to an Xpenology VM (so I can use the smartmontools directly from the VM).

I'm thinking about migrating from ESXi to SmartOS, but I have a few questions:

Is PCI passthrough not available through SmartOS? I saw another thread where it was said not to be the case. If it's not available, does anyone have another idea about how I can use Smartmontools with my Xpenology array?

Does anyone know if the X9SCL-F is compatible with SmartOS/Solaris? Any experience?

I also have an LSI 9212-4i (HP branded) I'm using for the datastores. Anyone know if there's compatibility issues there?

Most of all, how easy is it to migrate from ESXi (6.0) to SmartOS?

How's the FIFO interface compared to ESXi? I don't have enough room on my datastore to try it for myself right now ...

Thanks for all your help!
Avery
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
I've passed thru LSI HBA's in SmartOS so no issue there for sure for ZFS pools. As for 'how easy is it to migrate from ESXi to SmartOS you had better have a good grasp on SmartOS's KVM implementation as well as sorting out your migration path/tools of choice. Maybe a P2V tool of some sort.

Not sure what you mean by 'FIFO interface compared to ESXi'...stumped me there as well.
 

Avery Freeman

New Member
Aug 25, 2016
3
0
1
42
Hi Whitey,

Thanks for the reply. Sorry I wasn't clear - FIFO is the GUI that Joyent developed for SmartOS. I was wondering if anyone had used it and what they think of the GUI (interface) as compared to ESXi.

Thanks again!
Avery
 

knorrhane

New Member
Dec 17, 2016
14
0
1
43
Hi Whitey,

Thanks for the reply. Sorry I wasn't clear - FIFO is the GUI that Joyent developed for SmartOS. I was wondering if anyone had used it and what they think of the GUI (interface) as compared to ESXi.

Thanks again!
Avery
Hi! I'm in the process of doing the same. How did it turn out for you?

I installed SmartOS as a VM in ESXi first to familirize myself with FiFo and LeoFS. Right now I'm starting with the installation "for real".
 

knorrhane

New Member
Dec 17, 2016
14
0
1
43
That E3 system should be compatible.
Hi!

Yes, I just installed it on my Supermicro X10SLM+-LN4F with E3-1240 v3 and everything is working great.

I'm curious if it was possible to migrate the VMs from ESXi in to KVM in SmartOS so we'll see if OP responds.
 

Matthew Stott

New Member
Dec 28, 2016
8
4
3
52
There is an ESXi 4.0 explanation on the SmartOS Wiki but it may be far too outdated. It may however be enough to figure it out. Migrating from ESXi 4.x - SmartOS Documentation - SmartOS Wiki

If you are dealing with Linux VMs then you should know that the latest SmartOS does now have LX Branded Zones which are a normal Solaris Zone but with a translation layer within for running Linux 64bit binaries. It's so efficient that it runs Linux at full bare metal speed as opposed to the KVM which is very much like VMware. So it may be advantageous to create a new LX Branded Zone setup your configuration and merely migrate the data to it from a backup.

This is how Joyent has implemented Docker images at bare metal speed within SmartOS (Triton). On the full blown Joyent solution (requires a head node and a compute node) you can deploy Docker images to production in an extremely secure implementation and do it across many nodes and even multiple data centers. But setting up a head and compute node in a home lab is very much overkill unless you are planning on building an actual data center.
 

knorrhane

New Member
Dec 17, 2016
14
0
1
43
Yes, this is actually what I've been doing with the Ubuntu VMs I had and it's working really well. I really like how I easily I can mount a ZFS dataset into a LX zone using LOFS and I used that for setting up a SAMBA/CIFS server with Ubuntu.

I actually looked into Triton but as you say it was way to overkill for a home lab just looking at the requirements for running Triton.

Thanks for the reply!
 

Matthew Stott

New Member
Dec 28, 2016
8
4
3
52
ZFS needs as much RAM as you can throw at it. SmartOS / Triton needs hardware compatible with Illumos the kernel forked from Open Solaris. It's picky about Ethernet and SATA controllers. Recommend Intel and LSI based hardware. You do need all those fancy virtualization features and AMD will only partially work. You can't do KVM with AMD CPUs. Some consumer boards have bizarre chipsets but Supermicro is very compatible. Joyent has some hardware specs for their production compute nodes. But then you are looking at Xeon DP e5 CPU's with 24 cores and a minimum of 192GB RAM.

I run a low powered media server on SUPERMICRO MBD-A1SRi-2758F-O Mini ITX with Intel Atom C2758 8-core 20W. The PCI slot has an LSI 9211-8i flashed into JBOD mode. With 32GB RAM in a Norco ITX-8 bay hot swap case. The backplane offers two SAS cables to controller offering 8 SATA drives.

My previous 2012 build was Supermicro X9SCM-F with 32GB RAM Intel Xeon E3-1270 V2 Ivy Bridge 3.5GHz 79W and another LSI 9211-8i controller. The Atom C2758 is slower but keeps pace remarkably. It uses a heck of a lot less electricity.

My servers are idle most of the time so it matters what your specific needs are.
 

JayG30

Active Member
Feb 23, 2015
232
48
28
38
I feel like I (or somebody) should make video tutorials on setting up and testing all the things you can do with Triton. Been messing with it for the past few months and have worked through many of the initial steps I didn't understand. Like creating windows VM image/template, migrating running VM's, and setting up docker. Unfortunately I think I might be wiping the cluster now that I've gone through it and setup Proxmox or oVirt w/ gluster to play with that.
 

Matthew Stott

New Member
Dec 28, 2016
8
4
3
52
If you are setting up Triton open source with a Head and Compute node most of the Joyent documentation will get you pretty far. They have some tech they haven't open sourced internally that makes life a little easier. Some of the knowledge is Solaris need to know stuff that you can find at Oracle Solaris 10 documentation. You can ask questions on the smartos mailing list and I believe they have an IRC channel out there. They do sell support contracts with expert advice. There is a community out there but it's not been as active since they open sourced the whole schebang and started Docker work. That seems to be their current focus, is Docker as it's mighty popular with Dev's and DevOps. Orchestration, etc. is where it's at. The only Windows VMs would be legacy systems meant to be replaced by either Linux LX Zones or native Triton Zones. You can do a heck of a lot with shell scripts, Ruby/Python and Chef or Puppet. You can abstract out OS, Data, and Apps within a zone if you are smart about splitting up the storage and organizing things the right way. Then build a new VM and migrate the Data & Apps storage and re-attach it.