Has anyone got FreeNAS working on VMware ESXi ?

[JohnnyBeGood]

Yeah you don't need a vSwitch1 (you have no phys nics assigned/avail for it which equals completely isolated storage unless you ONLY want this host to connect to it then yes you could take this route but I am not EVEN muddying the waters w/ that discussion right now. Should be no issue to have vmk0 (mgmt) and vmk1 (vmkernel IP SAN conn) IP's assigned on same vSwitch0. If you lose mgmt, you can always reset very quickly via DCUI (yellow config screen). No need to re-install.

First of thanks for letting me know DCUI (yellow config screen) did not even knew it was there. If I knew about it it would save me countless reinstalls.

So I found my old PCI slot Intel PRO/1000 PT Dual Port Server Adapter that I use to use with pfSense and its recognized in ESXi:


I know its old and should used something newer like PCIe but I will worry about that once I get it working.

I tried to follow your above explanations on what to do when I have two physical nics but I kinda lost you at the end because I have nothing to reference it to.
Here's what I came up with. Do I need to physically connect chosen port on my Intel it to my router ?

 

[pricklypunter]

Nip over to the FreeNAS forum, tell em that you only have 8GB of RAM in the box and your sharing it with a windows VM...I dare ya

I wish I had $1 every time I heard someone complain about their forum and the folks that frequent it. What's amazing is that, even after all this time, despite folks complaining to the owners about their staff attitude on there, nothing is ever done about it. Now I don't know what being part of a forum means to you lot, that's for you to decide, but for me, it's a place where I can find and make some pals, maybe get some help when I need it and possibly help out some other folks by sharing my experiences. If I get to have a laugh or two along the way, even better. It's all about sharing knowledge and community spirit. I would have to say that on the FreeNAS forum, you'll find none or very little of that, unless your willing to berate any newcomer that has questions, just so you can be "in the club". It's a completely distasteful experience all round imo, I don't even like reading threads there.

 

[Keljian]

Nip over to the FreeNAS forum, tell em that you only have 8GB of RAM in the box and your sharing it with a windows VM...I dare ya

And this is one of the reasons I won't use Freenas.

And one of the key reasons I use pfsense, openmediavault and Ubuntu. All three forums have trolls like any forum, but generally you get a good amount of support, and they have a big enough install base.

I really want a COW filesystem for my server, with checksumming, but ZFS/freenas isn't going to be it for me. (Contemplating btrfs on mdadm..)

 

[whitey]

First of thanks for letting me know DCUI (yellow config screen) did not even knew it was there. If I knew about it it would save me countless reinstalls.

So I found my old PCI slot Intel PRO/1000 PT Dual Port Server Adapter that I use to use with pfSense and its recognized in ESXi:

View attachment 3037
I know its old and should used something newer like PCIe but I will worry about that once I get it working.

I tried to follow your above explanations on what to do when I have two physical nics but I kinda lost you at the end because I have nothing to reference it to.
Here's what I came up with. Do I need to physically connect chosen port on my Intel it to my router ?

View attachment 3038

You need to setup just two vSwitches, vSwitch0 for mgmt and vSwitch1 for IP SAN/NAS traffic. On vSwitch1 create a vmkernel port group for the Hypervisor to connect/have a data plane to proxy traffic from your hypervisor to the NAS. Should have a 192.168.1.x static IP out of your DHCP range. Then create a port group on that same vSwitch1 and I assume you are not using VST (Virtual Switch tagging) and instead EST (external switch tagging) so no vlan definition required. Then add (may already have one) a vnic to your freenas box on that newly created port group. Definitely need that phys nic off the dual port PCI intel nic conned/wired up to switch/router though. I can take screenshots if you get hung up.

Pssst, we don't need that vSwitch2 w/ vmnic2 connected but it is interesting that it is ONLY linked up at 10/100 (fast ethernet speeds) you will certainly want to sort that out and get it (or rather vmnic1 on vSwitch1) running at GigE speeds for your stg network.

 

[whitey]

Never did hear back abt your resources in that server or what we are working with? Please provide when you get a chance.

 

[CreoleLakerFan]

And this is one of the reasons I won't use Freenas.

And one of the key reasons I use pfsense, openmediavault and Ubuntu. All three forums have trolls like any forum, but generally you get a good amount of support, and they have a big enough install base.

I really want a COW filesystem for my server, with checksumming, but ZFS/freenas isn't going to be it for me. (Contemplating btrfs on mdadm..)

You don't have to use FreeNAS to use ZFS - there are alternatives. The only reason I still use FreeNAS at home is because I have used it a lot, and it's familiar. I never visit the forums anymore, but I did (years ago) before I realize what a bunch of cocks those guys were - which took all of a few visits.

I use Nexenta in my lab (VAAI), but I wouldn't use it at home because of the community licensing restrictions. One of these days I will migrate my home NAS over to Napp-IT, just haven't gotten around to it yet.

 

[whitey]

Yeah other than the asshat/holyier than thou attitude (I swear 'some' of the BSD communities are CLOSE to Oracle DBA bastard status)...maybe that's just me.

GREAT technology/capabilities/product though all in all. Super sick built in replication. I came from OmniOS and on base config (no napp-it) and I would occasionally get tcp buffer exhaustion or NFS svr daemon lockups/kernel panics (variety of these symptoms) a few times a year and finally jumped ship to FreeNAS and have yet to have a hiccup in several months so I am sold. Gotta admit though my cacti graphs and integrated FreeNAS graphs do show a 15-20% degradation in throughput in identical ZFS pool configs (between OmniOS and FreeNAS) for some odd reason but I personally will take stability over performance ANY day of the year.

I KNOW the Illumos core group is SOLID though so I dunno if it was my setup or what was going on. Very strange.

 

[herby]

Nip over to the FreeNAS forum, tell em that you only have 8GB of RAM in the box and your sharing it with a windows VM...I dare ya

I wish I had $1 every time I heard someone complain about their forum and the folks that frequent it. What's amazing is that, even after all this time, despite folks complaining to the owners about their staff attitude on there, nothing is ever done about it. Now I don't know what being part of a forum means to you lot, that's for you to decide, but for me, it's a place where I can find and make some pals, maybe get some help when I need it and possibly help out some other folks by sharing my experiences. If I get to have a laugh or two along the way, even better. It's all about sharing knowledge and community spirit. I would have to say that on the FreeNAS forum, you'll find none or very little of that, unless your willing to berate any newcomer that has questions, just so you can be "in the club". It's a completely distasteful experience all round imo, I don't even like reading threads there.

The assholery over there took me by surprise as an infrequent guest. I remember a great mod over there with the username ProtoSD. Always super helpful but at some point he took off and some new crop of really unpleasant people took over.

Still use FreeNAS mostly for the familiarity, wish I could get it running properly in Xenserver.

---

Xenservers
-big-red: Supermicro H8DCL-6F, 4332 HE (x2), 32GB ECC, USB boot, 6950
-lil-red: Supermicro H8SCM-F, 4376 HE, 16GB ECC, USB boot, 6450

FreeNAS
-blue: Supermicro H8SCM-F, 4376 HE, 16GB ECC, USB boot, 240GB M500 (x3), H310
-SE-3016: 1TB WD10EACS (x2), 2TB 5k3000 (x2), 3TB DT01ACA300 (x5)

 

[gbeirn]

The freenas forums are terrible imho. I setup a freenas VM on ESXi. ESXi passes through my LSI card with 8 Intel SSDs to freenas.

There is a virtual switch setup in ESXi with jumbo frames. There are no physical adapters bound to it. ESXi and freenas connect to it on a separate 10.x.x.x scheme purely for storage.

Freenas does the configuring of the drives passed to it and passes it back over this virtual network to ESXi as either a nfs share or iSCSI.

In my particular situation iSCSI performed much better.


Edit: I referenced that guide you posted. Didn't follow it exactly but yes that is basically what I ended up doing.

 

[whitey]

Yessir, this route works as well if you do not intend to share storage to anything else other than the hypervisor.

 

[Cole]

And this is one of the reasons I won't use Freenas.

And one of the key reasons I use pfsense, openmediavault and Ubuntu. All three forums have trolls like any forum, but generally you get a good amount of support, and they have a big enough install base.

I really want a COW filesystem for my server, with checksumming, but ZFS/freenas isn't going to be it for me. (Contemplating btrfs on mdadm..)

Have you looked into NAS4Free?

 

[Keljian]

Have you looked into NAS4Free?

The key feature that ZFS doesn't have, which I want, is the ability to grow or shrink an array (vdev) by adding/removing disks. Btrfs won't have this either when r5/6 is ready as far as I know.

 

[pricklypunter]

At some point in the future BTRFS will have this functionality I'm sure, it's much more community driven, but like all complicated things, it will take time, money and dedication on the part of the developers to bring it about and make it reliable. I very much doubt that Oracle will implement this in ZFS until they have something in the way of serious competition, and even then it would have to be something that the Enterprise made a fuss about I reckon. I think it's unlikely though, as their financial model differs radically from that of the home/ small business user, and it's large Enterprise users that Oracle mostly focus on. I keep my fingers crossed, but for now, ZoL works for me as is

 

[whitey]

Did this gent disappear on us? @JohnnyBeGood let us know where you stand.

 

[Keljian]

So I caved, I moved over to FreeNAS and ZFS (raidz2) under esxi- don't tell anyone I'm running non-ECC ram, windows VM, a few Linux VMs and other things on the box...

 

[pricklypunter]

Hehe, your secret is safe with me

 

[CreoleLakerFan]

So I caved, I moved over to FreeNAS and ZFS (raidz2) under esxi- don't tell anyone I'm running non-ECC ram, windows VM, a few Linux VMs and other things on the box...

Well, if you are running that under a VM, which you shouldn't be, and with no ECC, which you shouldn't be, I sure hope you're running at least 72 Quadzillion jigabytes of RAM!

[/sarcasm]

 

[whitey]

So I caved, I moved over to FreeNAS and ZFS (raidz2) under esxi- don't tell anyone I'm running non-ECC ram, windows VM, a few Linux VMs and other things on the box...

Bravo, except I worry about your non-ECC memory setup and corruption...'eventually/possibly'. Never had any issues but then again I only run ECC memory so i have never rolled the dice in that arena.

NFS or iSCSI, 1GbE or 10G?

 

[Keljian]

Smb, 10gig (chelsio t420-so-cr), 550MB/s over 8x7200rpm drives. Re memory corruption, the chances are extremely unlikely considering I have been running on mdraid for a long while and have not suffered corruption, and there is the multiple scrub thing- ie, one scrub corrupts some data, next one cleans etc.

Don't forget mdraid arrays should be scrubbed regularly too, and that could have the same theoretical issue.

 

[whitey]

Smb, ya lost me there, as in CIFS? VMware doesnt support that as a stg protocol, there must be a mixup?