[FreeNAS] 2 different disks sets 1 pool?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Operations

Member
Dec 29, 2019
78
2
8
I am running FreeNAS 11.3-U1. I have got 7x3TB on a Dell card flashed as a LSI 9207 and 8x4TB on another Dell card (also flashed).

I would like to use all the disks in 1 Windows share.

I thought i could create two vDevs (both Raidz1) in one pool, i hope i am explaining this right :)
But when i do that i have to select FORCE and i get a popup that it is not recommended. Is there a different way to do this or just ignore the warning? Or should i not do this?

I am going to use this FreeNAS as a backup from the backup (so Raidz1 is fine) with an 10Gbit connection. Is a windows share the way to go, or is ISCI better/faster? The other site is a RAID6 (Areca 1882) with 12x4TB.
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Whats the drive config for z1 that you're doing 6+1 in vdev1, 7+1 in vdev2?
Did you try 1 z1 with 14+1 drives (not recommended) ?
What does the warning msg say?

And iSCSI vs CIFS depends on how you want to access it...
 

Operations

Member
Dec 29, 2019
78
2
8
Yes i tried adding all the drives. The error i then get is "not the same size".

I know SCSI is block and SMB is file transfer. I will be backing up files. So that is a +1 for SMB but i thought when using 10Gbit it is recommened to use SCSI
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
you could add them in mirror pairs, that should work if the 2 vdev option did not work
 

markarr

Active Member
Oct 31, 2013
421
122
43
You can have drives of different sizes in zpools. The stipulations are that you have to add the same type of zfs and same size. So if you create a 7 disk raidz1 you can expand the pool by another 7 disk raidz1, not an 8 disk raidz1. IIRC you can not shrink a zpool.
 

Operations

Member
Dec 29, 2019
78
2
8
You can have drives of different sizes in zpools. The stipulations are that you have to add the same type of zfs and same size. So if you create a 7 disk raidz1 you can expand the pool by another 7 disk raidz1, not an 8 disk raidz1. IIRC you can not shrink a zpool.
Okee i will add another 3TB drive than to make it equal. Do you also have any ideas about ISCSI/SMB?
 

Operations

Member
Dec 29, 2019
78
2
8
Windows Server RAID6 12x4TB Terrascale on an Areca 1882 4GB. Connected with an 10Gbit Mellanox Connectx-3.

Is it wise to add an SSD for any apps? If i only use the NAS for file backup are there any useful apps, for this specific use case? I mean a bittorrent is a nice useful app but not in this case.

I am doubting to install FreeNAS on an 32GB usb stick and plug it onto my internal USB port. I have enough SSDs (and Sata ports) so it it not a budget issue.

The machine wil not be 24x7 powered on. I will power it on by IPMI, run the backup and turn it off. Is it okee for me to use the USB stick? I mean i know it is possible, you can even mirror it for safety. But as far as i know an SSD drive is best practice.
 
Last edited:

Spearfoot

Active Member
Apr 22, 2015
111
51
28
I am running FreeNAS 11.3-U1. I have got 7x3TB on a Dell card flashed as a LSI 9207 and 8x4TB on another Dell card (also flashed).

I would like to use all the disks in 1 Windows share.

I thought i could create two vDevs (both Raidz1) in one pool, i hope i am explaining this right :)
But when i do that i have to select FORCE and i get a popup that it is not recommended. Is there a different way to do this or just ignore the warning? Or should i not do this?

I am going to use this FreeNAS as a backup from the backup (so Raidz1 is fine) with an 10Gbit connection. Is a windows share the way to go, or is ISCI better/faster? The other site is a RAID6 (Areca 1882) with 12x4TB.
You can expand a pool with vdevs of different sizes and/or types (mirror, raid-z1, raid-z2) -- it's just not 'recommended practice'.

I advise you to use RAID-Z2 instead of -Z1 -- it's safer.

So in your case you want to create a pool with two vdevs: one made up of 7x3TB drives, the other made up of 8x4TB drives.

Try creating the pool in two steps:
  • Create a pool using your 7x3TB drives in RAID-Z2
  • Extend the pool adding your 8x4TB drives, also in RAID-Z2
 

Operations

Member
Dec 29, 2019
78
2
8
@Spearfoot, like i said earlier this is not possible (error: not the same size) and according to @markarr this is because i need the same number of disks. So i am waiting for a Icy-dock bracket. When this bracket arrives i will add another 3TB disk, so than i will have 8x3 and 8x4.

And it is not 'recommended practice' like you said, but it is not "wrong" or it will not break down at some point etc.? I have quit some experience with (hardware) RAID but Linux/FreeNAS is pretty new to me, so i am sorry if it is a bit of a dumb question :p
 
Last edited:

Spearfoot

Active Member
Apr 22, 2015
111
51
28
@Spearfoot, like i said earlier this is not possible (error: not the same size) and according to @markarr this is because i need the same number of disks. So i am waiting for a Icy-dock bracket. When this bracket arrives i will add another 3TB disk, so than i will have 8x3 and 8x4.

And it is not 'recommended practice' like you said, but it is not "wrong" or it will not break down at some point etc.? I have quit some experience with (hardware) RAID but Linux/FreeNAS is pretty new to me, so i am sorry if it is a bit of a dumb question :p
No need for apologies, and "there's no such thing as a dumb question".;)

Maybe I misunderstand what you mean when you say this is not possible.

You don't have to have the same number of disks in every vdev making up a pool -- you can have 7 in one and 8 in the other, as I described before. It won't break anything to do this; it's just not an 'optimal' design.

Granted, it's a bad idea to use different size disks in the same vdev. If you do this, you're restricted to the size of the smallest disk in the group. That's wasteful, so you do want to put your 3TB and 4TB disks in separate vdevs.

I've created oddball pools with vdevs of different types before, just for grins -- so I know that it can be done. And I run a few systems at work with pools made up of vdevs with different-sized disks. For example: one system has a pool with one vdev of 8 x 10TB disks in RAIDZ2 and another vdev of 8 x 12TB disks in RAIDZ2. Granted, both vdevs have 8 disks, but that's because it's a 16-disk chassis. FreeNAS doesn't care if one vdev has 7 disks and another has 8.

One of the common problems we see on the FreeNAS forum goes something like this: "Help! I've extended my 8-disk RAIDZ2 pool with a single striped disk! How do I fix it?" Sadly the answer is: "Sorry, pal, you're going to have to tear down the pool, rebuild it, and restore from backup. Once you extend a pool, there's no going back."

Not trying to discourage you from buying another drive and Icy Dock goodies. I love gadgetry too! And frankly, while the older FreeNAS pool management GUI was pretty tough, at least you could wade your way through it. The newer version 11.3 GUI is absolutely horrible. Very easy even for an old hand to goof things up. Ask me how I know. Anyway, I'm still running 11.2U8.

Can you post a screen shot of the error message FreeNAS shows you?
 

Operations

Member
Dec 29, 2019
78
2
8
No need for apologies, and "there's no such thing as a dumb question".;)

Maybe I misunderstand what you mean when you say this is not possible.

You don't have to have the same number of disks in every vdev making up a pool -- you can have 7 in one and 8 in the other, as I described before. It won't break anything to do this; it's just not an 'optimal' design.

Granted, it's a bad idea to use different size disks in the same vdev. If you do this, you're restricted to the size of the smallest disk in the group. That's wasteful, so you do want to put your 3TB and 4TB disks in separate vdevs.

I've created oddball pools with vdevs of different types before, just for grins -- so I know that it can be done. And I run a few systems at work with pools made up of vdevs with different-sized disks. For example: one system has a pool with one vdev of 8 x 10TB disks in RAIDZ2 and another vdev of 8 x 12TB disks in RAIDZ2. Granted, both vdevs have 8 disks, but that's because it's a 16-disk chassis. FreeNAS doesn't care if one vdev has 7 disks and another has 8.

One of the common problems we see on the FreeNAS forum goes something like this: "Help! I've extended my 8-disk RAIDZ2 pool with a single striped disk! How do I fix it?" Sadly the answer is: "Sorry, pal, you're going to have to tear down the pool, rebuild it, and restore from backup. Once you extend a pool, there's no going back."

Not trying to discourage you from buying another drive and Icy Dock goodies. I love gadgetry too! And frankly, while the older FreeNAS pool management GUI was pretty tough, at least you could wade your way through it. The newer version 11.3 GUI is absolutely horrible. Very easy even for an old hand to goof things up. Ask me how I know. Anyway, I'm still running 11.2U8.

Can you post a screen shot of the error message FreeNAS shows you?
I will show you the error asap.
Since you played around with exotic pools, what would the performance be like when i combine the 2 vDEVs? Will it be okee or severe write penalties, and will i be better of creating two pools?

Do you also have any thoughts on SCSI/SMB and using an extra SSD yes or no?

I am running 32GB of memory, that shouldn't be a problem performance wise right? Or will it help if i add a SSD as cache?
 
Last edited:

Spearfoot

Active Member
Apr 22, 2015
111
51
28
I will show you the error asap.
Since you played around with exotic pools, what would the performance be like when i combine the 2 vDEVs? Will it be okee or severe write penalties, and will i be better of creating two pools?

Do you also have any thoughts on SCSI/SMB and using an extra SSD yes or no?

I am running 32GB of memory, that shouldn't be a problem performance wise right? Or will it help if i add a SSD as cache?
We're talking about creating one pool. I think there's been some confusion about that. :)

Pools are made up of one or more virtual devices ('vdevs'). A virtual device can be a single disk; 2 (or more!) disks in a mirror;
or multiple disks in a RAIDZ1/2/3 configuration.

This one pool will have two vdevs: one vdev with 7 x 3TB drives (or 8, when the new disk you ordered arrives), and one with 8 x 4TB disks.

In terms of performance: the more vdevs the better, because IOPS scale by the number of vdevs. Your most performant config would be a bunch mirrors; but you'd lose a lot of capacity doing that.

You say this is a 'backup of a backup', but I still advise against RAIDZ1 -- it's just not safe.

I run several FreeNAS servers at work with 32GB of RAM and 16 disks in two 8-disk RAIDZ2 vdevs, with disk sizes ranging up to 16 x 10TB; no SSD for L2ARC cache; and using SMB. They're not speed demons, but they serve their job as backup storage, which is all we use them for.

If you were to go the iSCSI route, you'd want a lot more memory, plus block storage is more difficult to work with -- you can't just snag an individual file off the server like you can with Samba.
 

Operations

Member
Dec 29, 2019
78
2
8
We're talking about creating one pool. I think there's been some confusion about that. :)

Pools are made up of one or more virtual devices ('vdevs'). A virtual device can be a single disk; 2 (or more!) disks in a mirror;
or multiple disks in a RAIDZ1/2/3 configuration.

This one pool will have two vdevs: one vdev with 7 x 3TB drives (or 8, when the new disk you ordered arrives), and one with 8 x 4TB disks.

In terms of performance: the more vdevs the better, because IOPS scale by the number of vdevs. Your most performant config would be a bunch mirrors; but you'd lose a lot of capacity doing that.

You say this is a 'backup of a backup', but I still advise against RAIDZ1 -- it's just not safe.

I run several FreeNAS servers at work with 32GB of RAM and 16 disks in two 8-disk RAIDZ2 vdevs, with disk sizes ranging up to 16 x 10TB; no SSD for L2ARC cache; and using SMB. They're not speed demons, but they serve their job as backup storage, which is all we use them for.

If you were to go the iSCSI route, you'd want a lot more memory, plus block storage is more difficult to work with -- you can't just snag an individual file off the server like you can with Samba.
@Spearfoot ,

Yes i would like to use 1 pool, but if 2 pools (and so use two SMB connections instead of one on the Windows Server) would be much faster i might still do this. I don't like splitting the backup in "two" parts, but if it is much faster i might still do it. With 2 pools, i mean splitting up the 8x3TB and 8x4TB sets. With one pool, i mean 8x3TB in a vDEV and the 8x4TB in a vDEV. I hope there is no more confusing about this :) And yes i realize the confusing is most likely my fault :p

Why is Raidz1 not safe? i think you are talking about the long rebuild (different word in ZFS, right?) and the stress on the other disks and the chance of another one failing in the rebuild period. Is this your reason why?

My Primairy backup is RAID6 (as you probably know is similair to RAIDz2), so it is not that i do not agree with you. But backup of a backup and 8x4 and 8x3, "giving up" 4 disks is kinda beating the purpose of backup from the backup since it gives me not enough space anymore.

Are there anymore reasons for switching to RaidZ2? I will think about it a bit more.

You say "no speed demons" but what are me talking about in terms of MB/s? my RAID6 can do around 700-850MB/s over the 10Gbit network. You didn't use a L2ARC cache, why not and would it have helped in your case? And what about my case?

I am using FreeNAS because i think it is fun and is probably a good system. But..... if between Windows Servers (and hardware RAID) will give me much more throughput speed i might have made the wrong choice... Not a huge problem could easily switch to Windows Server, but from a tech POV i wanted to use FreeNAS :)
 
Last edited:

Operations

Member
Dec 29, 2019
78
2
8
@Spearfoot,

I have added my extra disk now, so it is 8x3TB and 8x4TB. But how do i create 2 vDEVs in one pool? What i do now is i go to Pools, click add pool. Then i see the 16 disks. when i add them all it shows the error of "not the same size" but i can force it te continue. But when it is done i have got 1 vDEV in a pool with 16 disks.

When i go to pools and add both sets of disks seperate then i end up with 2 pools.
 

markarr

Active Member
Oct 31, 2013
421
122
43
You would create a pool with one set of disks and then once that is created then you will expand that pool and that will allow you to make another vDev.
 

Operations

Member
Dec 29, 2019
78
2
8
You would create a pool with one set of disks and then once that is created then you will expand that pool and that will allow you to make another vDev.
All like that. What would happen to the pool if one of both sets would have problems? Would the whole pool be unavailable? And/or loss of data?

I will try to create the pool the way you described :)
 

markarr

Active Member
Oct 31, 2013
421
122
43
So if you were to loose one vDev yes the whole pool will become unavailable, and you will have data loss. This is why even though it is "wasteful" people were suggesting raidz2.
 

Operations

Member
Dec 29, 2019
78
2
8
So if you were to loose one vDev yes the whole pool will become unavailable, and you will have data loss. This is why even though it is "wasteful" people were suggesting raidz2.
Isn't better to create a pool with one vdev? 16x3TB? In Raidz2 i will have 14x3TB space.

No i don't have 16 x 3TB disks, so the 4TB disks will be used as 3TB disks.