opinions and assistance please

dragonme

Active Member
Apr 12, 2016
295
29
28
I have a fairly old esxi home lab based on intel s5520HC and 2x L5640s

the setup serves my purposes and upgrading the hardware is not in the cards at the moment..

I am on 6.0xx still as initial testing when 6.5 came out that power optimization was not as good and my 24x7 server that mostly sees light load was using more wall power at idle.. I dont know if its improved or not with 6.7 or latest updates.. 7.0 is a non starter based on hardware compat issues

I have this server in AIO configuration with Napp-it home use and omnios-r151022-f9693432c2

one issue is that a pool that I had been using for many years, had moved into this setup as a 2 device pool, and using RDM.. that I later added a device to, also using RDM... it was working fine and reported that it was using the whole disk..

strangely when I changed the configuration from RDM to passing the whole SATA controller to Napp-it.. the 3rd drive that was added shows it using a partition and not the whole disk like the other 2.. very strange.. and pool performance is taking a hit..

this pool is 3 x 8TB and it at 70%ish filled... I have a backup using send/receive on a per dataset level to a shelf that has 3 x 5 wide raidz1 devs

FIRST QUESTION (finally right)
to correct the pool, I am deciding between the option of doing a replace operation using a new 8TB drive and let is silver or to destroy the pool create and restore it from scratch...

the drawback to the first is that resilver takes a long time and is rather thrashy

the drawback to destroy rebuild is that I would be down to just one copy of roughly 18TB of info... albeit with redundancy but more than a single drive failure in either of the 3 raidz luns would result in loss...

the replace operation seems pretty straight forward but if I go the destroy and use send/rcv route.. what napp-it sequence is best .. do the send one dataset at a time? do the entire pool? and can I, after the restore, go back to using the backup pool with the existing jobs without having to rebuild the backup.. ?


SECOND QUESTION

having to stay at ESXI 6 .. should I just keep using omni 22 or update to 36? the upgrade path would require updating omni multiple times as a straight upgrade from 22 to 36 is not supported... so building a new VM I would have to install 36 from scratch and then install napp-it and reconfigure from scratch since the pre-build VM is for later ESXI versions ?

recommendations here ?
 

gea

Well-Known Member
Dec 31, 2010
2,589
879
113
DE
The straight forward method woul dbe add a new disk and start a disk > replace to avoid a degration, then remove the old disk. Of course you can add a new pool and replicate.

I would use the newest OmniOS as there are a lot of critical bugs fixed (ex OpenSSL) and SMB is now on 3.1.1 while it was on 2.1 on OmniOS 151022. Update from 022 > 030 > 036 is a little tricky as you may find problems due old packages like gcc no longer supported (must be removed prior an update) or the switch from Sun SSL to OpenSSL. As an alternative, install a current OS (151030, 034, 036) and import the pool, see http://www.napp-it.org/doc/downloads/setup_napp-it_os.pdf

To keep user and napp-it settings after a reinstall of a current OmniOS
- install vmware tools (pkg install open-vm-tools)
- recreate all users with same uid/gid
- save/ restore /var/web-gui/_log/*
 

dragonme

Active Member
Apr 12, 2016
295
29
28
The straight forward method woul dbe add a new disk and start a disk > replace to avoid a degration, then remove the old disk. Of course you can add a new pool and replicate.

I would use the newest OmniOS as there are a lot of critical bugs fixed (ex OpenSSL) and SMB is now on 3.1.1 while it was on 2.1 on OmniOS 151022. Update from 022 > 030 > 036 is a little tricky as you may find problems due old packages like gcc no longer supported (must be removed prior an update) or the switch from Sun SSL to OpenSSL. As an alternative, install a current OS (151030, 034, 036) and import the pool, see http://www.napp-it.org/doc/downloads/setup_napp-it_os.pdf

To keep user and napp-it settings after a reinstall of a current OmniOS
- install vmware tools (pkg install open-vm-tools)
- recreate all users with same uid/gid
- save/ restore /var/web-gui/_log/*
thanks Gea...

buying new drives to make a 3rd pool to replicate to I cant afford right now.. and I dont hear you advocating destroying and restoring the current pool from the single backup.. so I guess you are advocating the disk replacement method... question is to why did Napp-it, when this third disk was added through the gui add it using a partition and not the whole drive and will it just do it again with this new drive?

you are saying do an add then replace ... I dont think that is right. because if I add the 4th drive to the existing 3 drive pool, it will not permit me to just remove the partitioned 3rd drive since there is no redundancy its just a straight pool

wouldn't the proper way be to just insert the new drive, verify that Napp-it sees it as an unattached available volume, then execute a replace

pool: onlinepool
state: ONLINE
scan: scrub repaired 0 in 19h3m with 0 errors on Wed Dec 9 23:14:18 2020
config:

NAME STATE READ WRITE CKSUM CAP Product /napp-it SN/LUN IOstat mess SMART
onlinepool ONLINE 0 0 0
c13t3d0 ONLINE 0 0 0 8 TB WDC WD80EZZX-11C VKGJUTLX S:0 H:0 T:0 -
c13t2d0 ONLINE 0 0 0 8 TB WDC WD80EZZX-11C VKGNLVYX S:0 H:0 T:0 -
c13t4d0s1 ONLINE 0 0 0 -

for example if the new drive ends up at c13t5d0 just do a replace c13t4d0a1 with c13t5d0 and once resilver is accomplished, the faulted drive would no longer be assigned to the pool, could be re-formatted and attached as a 4 dev to to the pool?

in the above pool, see how napp-it attached it in the first place>?

what is even stranger is even though that disk with its one and only partition are part of onlinepool, napp-it still shows the parent disk dev as available...

so when I insert a new disk into the backplane how do I ensure its formatted correctly and will be attached as a whole disk and not a jacked up and added like a partition again like it did that time?
 

Attachments

gea

Well-Known Member
Dec 31, 2010
2,589
879
113
DE
ZFS on Solarish always use and prefer whole disks as default unless you force using partitions. (Add a partition instead a whole disk like c1t0d0). Only bootmirrors can behave different per default.

No need to prepare new disks or disks with a single partition.
Action (delete partitions) is only needed on protected partitions or if a disk has already multiple partitions.
 
Last edited:

dragonme

Active Member
Apr 12, 2016
295
29
28
ZFS on Solarish always use and prefer whole disks as default unless you force using partitions. (Add a partition instead a whole disk like c1t0d0). Only bootmirrors can behave different per default.

No need to prepare new disks or disks with a single partition.
Action (delete partitions) is only needed on protected partitions or if a disk has already multiple partitions.
so then how did napp-it default to adding this drive by partition? I just don't get it...

I normally have used ZFS completely by command line... so I know exactly what I am doing and how I am doing it...

what I would like to see with napp-it.. either by default or as an optional setting ... is for it to show me the EXACT command its going to issue and ask for confirmation before executing it.. also, I would like to see napp-it LOG EVERY COMMAND it issues.. whether its a cron job, user directed, etc

so while I KNOW that zfs should be using whole drives.. and when I set that pool up on another system manually with the first 2 drives.. you can see it used whole drives.. WHY DID NAPP IT FAIL.

further to my question .. how can I ensure Napp-it formats and attaches the replacement drive correctly .....
 

gea

Well-Known Member
Dec 31, 2010
2,589
879
113
DE
Napp-it does nothing by its own regarding partitions. When you create or extend a pool it shows a list of disks or partitions and you can select either from the list.

Without partition support (that you can enable in menu Disks > Partitions) napp-it just reads in disks via a format command (that shows only whole disks) and allows a zpool create/replace command with these disks selected. There is no partition or any other command involved. Whole disks are like c0t0d0.

If you enable partition support, you can create partitions ex c0t0d0p1. They are detected then via parted and you can select them for a zpool create/replace etc additionally to whole disks.

If you want to check what napp-it is doing on a zpool create, see menu action file "/var/web-gui/data/napp-it/zfsos/06_Pools/01_Create pool=-lin/action.pl"

There is no log file for every napp-it menu command. To check history of a pool, see menu Pools > History that shows what you have done with a pool.

You can see last menu command in the bottom of a page (mini log) when you point the mouse to the bottom of a page. With napp-it pro or an active evaluation you can enable top level menu "Edit". This allows to show the script action of a menu or you can click on menu item Log that shows an overview of what happened during menu processing.

Napp-it does not use Solaris slices like c0t0d0s1 beside the boot mirror process where they may be created automatically by the boot mirror process.