Not possible to import pool:...
If there is no change, export + import the pool readonly and try to backup changed data (asume you already have a backup) or as much as possible.
# zpool import
pool: tank1
id: 14720958912406048058
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-3C
config:
tank1 UNAVAIL insufficient replicas
mirror-0 ONLINE
c2t5000CCA23B0CEF7Dd0 ONLINE
c2t5000CCA23B0D18F9d0 ONLINE
mirror-1 DEGRADED
c2t5000CCA23B0CDAE9d0 ONLINE
c2t5000CCA23B0D0E11d0 UNAVAIL cannot open
mirror-2 DEGRADED
c2t5000CCA23B0C20C9d0 ONLINE
c2t5000CCA23B0CA94Dd0 UNAVAIL cannot open
mirror-3 ONLINE
c2t5000CCA23B07B701d0 ONLINE
c2t5000CCA23B0C9CD5d0 ONLINE
mirror-4 UNAVAIL insufficient replicas
c2t5000CCA23B0BE229d0 UNAVAIL cannot open
c2t5000CCA23B0C0935d0 UNAVAIL cannot open
mirror-5 DEGRADED
c2t5000CCA23B0BFDA9d0 ONLINE
c9t5000CCA23B0D25C9d0 UNAVAIL cannot open
mirror-6 ONLINE
c2t5000CCA23B0B9121d0 ONLINE
c2t5000CCA23B0BFCA1d0 ONLINE
mirror-7 DEGRADED
c2t5000CCA23B0BDA41d0 ONLINE
c9t5000CCA23B0BFBF1d0 UNAVAIL cannot open
mirror-8 ONLINE
c2t5000CCA23B0CE5B9d0 ONLINE
c2t5000CCA23B0CE7A9d0 ONLINE
mirror-9 UNAVAIL insufficient replicas
c2t5000CCA23B0C0901d0 UNAVAIL cannot open
c2t5000CCA23B0D1BB5d0 UNAVAIL cannot open
mirror-10 DEGRADED
c9t5000CCA23B0C00B1d0 UNAVAIL cannot open
c2t5000CCA23B0C9BD5d0 ONLINE
mirror-11 DEGRADED
c9t5000CCA23B0A3AE9d0 UNAVAIL cannot open
c2t5000CCA23B0CF6D9d0 ONLINE
logs
mirror-12 ONLINE
c1t5002538C401C745Fd0 ONLINE
c1t5002538C401C7462d0 ONLINE
# zpool import -f tank1
cannot import 'tank1': no such device in pool
Destroy and re-create the pool from
a backup source.
# zpool destroy -f tank1
cannot open 'tank1': no such pool
Yes and I have identified 11 disk in maximum and unfortunately two mirrored vdevs involved.Your pool state seems different from test to test. Only the result "UNAVAIL" remains as in all cases different disks are not available, in all cases more than the redundancy level allows.
Already done, pls. check above.A single point of failure that can introduce this is the HBA and the Expander. If possible replace the HBA.
I read about it, but with the limitation of SATA drives on SAS Expander. I only have SAS disks here.As there is a slight chance that a single bad disk irritates the expander, it may be an option to remove all disks, then insert disk by disk, wait a short time and check pool state if the disk came up. There is a slight chance that suddenly the pool/other disks change state when adding a disk (a bad disk then).
Completely agree, and I have had this issue: fmd fault after my update OmniOSce-r151026 -> OmniOSce-r151028. Not really sure currently if it has much more side effects regarding SAS HBA. What do you mean?One idea would be to use a new bootdisk, install OmniOS and check disk and pool state (via format and zpool status). Not very propably but the OS may be corrupted.
If you have a different case/jbod with enough empty bays, I would move all disks to the second system. Your disk behaviour is so unsystematic that I would suppose you have either an HBA/Expander problem or not only one problem but two overlaying problems in your server. If the pool is ok there, then one or propably some parts of the server are damaged.
pool: tank1
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
tank1 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c2t5000CCA23B0D18F9d0 ONLINE 0 0 0
c2t5000CCA2676CFAF9d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c2t5000CCA23B0CDAE9d0 ONLINE 0 0 0
c2t5000CCA2674A826Dd0 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
c2t5000CCA23B0CEF7Dd0 ONLINE 0 0 0
c2t5000CCA26767B28Dd0 ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
c2t5000CCA23B0C9CD5d0 ONLINE 0 0 0
c2t5000CCA2674D9035d0 ONLINE 0 0 0
mirror-4 ONLINE 0 0 0
c2t5000CCA23B07B701d0 ONLINE 0 0 0
c2t5000CCA2674A8D3Dd0 ONLINE 0 0 0
mirror-5 ONLINE 0 0 0
c2t5000CCA23B0BFDA9d0 ONLINE 0 0 0
c2t5000CCA2674938CDd0 ONLINE 0 0 0
mirror-6 ONLINE 0 0 0
c2t5000CCA23B0BFCA1d0 ONLINE 0 0 0
c2t5000CCA2674908F9d0 ONLINE 0 0 0
mirror-7 ONLINE 0 0 0
c2t5000CCA23B0BDA41d0 ONLINE 0 0 0
c2t5000CCA2674B31D9d0 ONLINE 0 0 0
mirror-8 ONLINE 0 0 0
c2t5000CCA23B0CE7A9d0 ONLINE 0 0 0
c2t5000CCA257925BE1d0 ONLINE 0 0 0
mirror-9 ONLINE 0 0 0
c2t5000CCA23B0B9121d0 ONLINE 0 0 0
c2t5000CCA257936939d0 ONLINE 0 0 0
mirror-10 ONLINE 0 0 0
c2t5000CCA23B0C9BD5d0 ONLINE 0 0 0
c2t5000CCA267407661d0 ONLINE 0 0 0
mirror-11 ONLINE 0 0 0
c2t5000CCA23B0CE5B9d0 ONLINE 0 0 0
c2t5000CCA2674D78F1d0 ONLINE 0 0 0
mirror-12 ONLINE 0 0 0
c2t5000CCA26742E4CDd0 ONLINE 0 0 0
c2t5000CCA267478921d0 ONLINE 0 0 0
mirror-13 ONLINE 0 0 0
c2t5000CCA26746049Dd0 ONLINE 0 0 0
c2t5000CCA2674CDF71d0 ONLINE 0 0 0
mirror-14 ONLINE 0 0 0
c2t5000CCA2673E3DD9d0 ONLINE 0 0 0
c2t5000CCA267460489d0 ONLINE 0 0 0
mirror-15 ONLINE 0 0 0
c2t5000CCA2674488F1d0 ONLINE 0 0 0
c2t5000CCA26744F179d0 ONLINE 0 0 0
logs
mirror-16 ONLINE 0 0 0
c1t5002538C401C745Fd0 ONLINE 0 0 0
c1t5002538C401C7462d0 ONLINE 0 0 0
cache
c3t1d0 ONLINE 0 0 0
spares
c2t5000CCA2676BC9F5d0 AVAIL
c2t5000CCA2676CF405d0 AVAIL
c2t5000CCA23B0CF6D9d0 AVAIL
errors: No known data errors
I was affected with a series of 3TB Seagate disks a few years ago where I nearly lost a Z3 backup pool. After around two/three years they died like flies. There was a rumour that it was due the air filter allowing dust into the internal after this time.Finally the problem was solved by replacing all 11 faulted drives with new one from myself in warranty. Don't believe it...
Is there anyone with similar failure rate of 11 in 24 drives after three years ins use?