OmniOS + NappIT VM: Major Fault (kernel.panic)/High CPU/Hard + Soft Smart errors
I'm concerned since i have very high CPU usage 50-75% without much of any SMB or NFS Access. SMB seams fast but NFS really drags. For example, I got over 200 MB/s by SMB and just 20 MB/s copying that same large file from SMB to a Win7 VM (on that same ZPool, by ESXI NFS Datastore). Everything is on a 6x 3TB WD Red NAS drives in RAID-Z2 + Kingston V300 120GB L2ARC. I'm moving some VMs to a separate 256GB SSD in ESXI.
I also see these two faults with SEVERITY=major in the logs below. The link does not help so I am uncertain how to troubleshoot. Any suggestions?
Running nappit-14a on ESXI 5.5, default 2 vCPUs and 8GB RAM.
fmadm faulty
I'm concerned since i have very high CPU usage 50-75% without much of any SMB or NFS Access. SMB seams fast but NFS really drags. For example, I got over 200 MB/s by SMB and just 20 MB/s copying that same large file from SMB to a Win7 VM (on that same ZPool, by ESXI NFS Datastore). Everything is on a 6x 3TB WD Red NAS drives in RAID-Z2 + Kingston V300 120GB L2ARC. I'm moving some VMs to a separate 256GB SSD in ESXI.
I also see these two faults with SEVERITY=major in the logs below. The link does not help so I am uncertain how to troubleshoot. Any suggestions?
Running nappit-14a on ESXI 5.5, default 2 vCPUs and 8GB RAM.
fmadm faulty
Stats--------------- ------------------------------------ -------------- ---------
TIME EVENT-ID MSG-ID SEVERITY
--------------- ------------------------------------ -------------- ---------
Jan 28 18:37:22 2380911a-891b-461b-dedc-c2b798451062 SUNOS-8000-KL Major
Host : napp-it-14a
Platform : VMware-Virtual-Platform Chassis_id : VMware-56-4d-e1-e1-51-f8-cf-cc-2f-df-7c-42-1d-4c-79-fb
Product_sn :
Fault class : defect.sunos.kernel.panic
Affects : sw:///ath=/var/crash/unknown/.2380911a-891b-461b-dedc-c2b798451062
faulted but still in service
Problem in : sw:///
faulted but still in service
Description : The system has rebooted after a kernel panic. Refer to
SUNOS-8000-KL for more information.
Response : The failed system image was dumped to the dump device. If
savecore is enabled (see dumpadm(1M)) a copy of the dump will be
written to the savecore directory /var/crash/unknown.
Impact : There may be some performance impact while the panic is copied to
the savecore directory. Disk space usage by panics can be
substantial.
Action : If savecore is not enabled then please take steps to preserve the
crash image.
Use 'fmdump -Vp -u 2380911a-891b-461b-dedc-c2b798451062' to view
more panic detail. Please refer to the knowledge article for
additional information.
--------------- ------------------------------------ -------------- ---------
TIME EVENT-ID MSG-ID SEVERITY
--------------- ------------------------------------ -------------- ---------
Jan 21 20:14:42 1178e515-dd0b-ed91-bdd8-ab0a28eb5f4f SUNOS-8000-KL Major
Host : napp-it-14a
Platform : VMware-Virtual-Platform Chassis_id : VMware-56-4d-e1-e1-51-f8-cf-cc-2f-df-7c-42-1d-4c-79-fb
Product_sn :
Fault class : defect.sunos.kernel.panic
Affects : sw:///
faulted but still in service
Problem in : sw:///
faulted but still in service
Description : The system has rebooted after a kernel panic. Refer to
SUNOS-8000-KL for more information.
Response : The failed system image was dumped to the dump device. If
savecore is enabled (see dumpadm(1M)) a copy of the dump will be
written to the savecore directory /var/crash/unknown.
Impact : There may be some performance impact while the panic is copied to
the savecore directory. Disk space usage by panics can be
substantial.
Action : If savecore is not enabled then please take steps to preserve the
crash image.
Use 'fmdump -Vp -u 1178e515-dd0b-ed91-bdd8-ab0a28eb5f4f' to view
more panic detail. Please refer to the knowledge article for
additional information.
Stat: fmstat
module ev_recv ev_acpt wait svc_t %w %b open solve memsz bufsz
cpumem-retire 0 0 0.0 0.0 0 0 0 0 0 0
disk-lights 0 0 0.0 0.1 0 0 0 0 28b 0
disk-transport 0 0 0.0 18.7 0 0 0 0 32b 0
eft 0 0 0.0 0.0 0 0 0 0 1.3M 0
ext-event-transport 3 0 0.0 6.4 0 0 0 0 46b 0
fabric-xlate 0 0 0.0 0.0 0 0 0 0 0 0
fmd-self-diagnosis 14 0 0.0 0.6 0 0 0 0 0 0
io-retire 0 0 0.0 0.0 0 0 0 0 0 0
sensor-transport 0 0 0.0 0.5 0 0 0 0 32b 0
ses-log-transport 0 0 0.0 0.2 0 0 0 0 40b 0
software-diagnosis 0 0 0.0 0.0 0 0 0 0 316b 0
software-response 0 0 0.0 0.0 0 0 0 0 2.3K 2.0K
sysevent-transport 0 0 0.0 389.2 0 0 0 0 0 0
syslog-msgs 0 0 0.0 0.0 0 0 0 0 0 0
zfs-diagnosis 15 0 0.0 0.7 0 0 0 0 0 0
zfs-retire 15 0 0.0 1.2 0 0 0 0 168b 0
Important: fmdump -I
TIME CLASS
Jan 11 20:12:19.2634 ireport.os.sunos.panic.savecore_failure
Jan 11 20:12:45.7120 resource.sysevent.EC_datalink.ESC_datalink_phys_add
Jan 11 20:19:17.9062 ireport.os.sunos.panic.savecore_failure
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_static_start
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_static_end
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_send_targets_start
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_send_targets_end
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_slp_start
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_slp_end
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_isns_start
Jan 11 20:22:59.3006 resource.sysevent.EC_iSCSI.ESC_isns_end
....(cut)
Last edited:
