troubleshooting vmware

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
Hi All,

I been working on my storage for home server. While working on it, i been upgrading vms, patching servers firmware, etc. Now one of main server, my windows server 2012 r2 essentials vm is hanging. Seems fine for a few hours then it just completely hangs. I need to reboot the host to restart it. the normal kill methods dont do it.

I made a few changes, and dont know which one may be causing the issue.

* I upgraded the hardware level to 6.5 (vmware version 13)
* I removed a passthrough ibm m1015 raid card which had 2x wd red 6tb nas drives on it. One was used for data the other for back up.
Now I'm using an intel raid card with 1GB cache, and ssd cachecade 2 enabled. I have two 8tb red nas drives configured as solo raid 0 and created a datastore for each in esxi host. I then created 2 vmfs drives and added it to windows vm.

Here are some snipets of vmware.log when i started it up. I havent done much vm log troubleshooting so hoping someone out there has :) and can help out.

I'll add more logs the next time it crashes.

Code:
vcpu-2| W115: LSI: ProcessIOCPageRequest: unsupported page 5 action 0
2017-08-03T03:40:05.944Z| vcpu-1| W115: LSI: ProcessIOCPageRequest: unsupported page 5 action 0
2017-08-03T03:40:05.945Z| vcpu-1| W115: LSI: ProcessIOCPageRequest: unsupported page 5 action 0
2017-08-03T03:40:05.945Z| vcpu-1| W115: LSI: ProcessIOCPageRequest: unsupported page 5 action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
2017-08-03T03:40:05.948Z| vcpu-1| I125: LSI: Invalid PageType [8] pageNo 0 Action 0
Code:
2017-08-03T03:40:15.771Z| vmx| I125: GuestRpc: Got error for channel 1 connection 6: Remote disconnected
2017-08-03T03:40:15.785Z| vmx| I125: GuestRpc: Got error for channel 1 connection 7: Remote disconnected
2017-08-03T03:40:16.858Z| vmx| I125: GuestRpc: Got error for channel 1 connection 8: Remote disconnected
2017-08-03T03:40:17.858Z| vmx| I125: GuestRpc: Got error for channel 1 connection 9: Remote disconnected
2017-08-03T03:40:18.874Z| vmx| I125: GuestRpc: Got error for channel 1 connection 10: Remote disconnected
2017-08-03T03:40:19.874Z| vmx| I125: GuestRpc: Got error for channel 1 connection 11: Remote disconnected
2017-08-03T03:40:20.875Z| vmx| I125: GuestRpc: Got error for channel 1 connection 12: Remote disconnected
2017-08-03T03:40:21.874Z| vmx| I125: GuestRpc: Got error for channel 1 connection 13: Remote disconnected
2017-08-03T03:40:22.890Z| vmx| I125: GuestRpc: Got error for channel 1 connection 14: Remote disconnected
2017-08-03T03:40:23.905Z| vmx| I125: GuestRpc: Got error for channel 1 connection 15: Remote disconnected
2017-08-03T03:40:24.905Z| vmx| I125: GuestRpc: Got error for channel 1 connection 16: Remote disconnected
2017-08-03T03:40:25.907Z| vmx| I125: GuestRpc: Got error for channel 1 connection 17: Remote disconnected
2017-08-03T03:40:26.906Z| vmx| I125: GuestRpc: Got error for channel 1 connection 18: Remote disconnected
2017-08-03T03:40:27.906Z| vmx| I125: GuestRpc: Got error for channel 1 connection 19: Remote disconnected
2017-08-03T03:40:28.908Z| vmx| I125: GuestRpc: Got error for channel 1 connection 20: Remote disconnected
2017-08-03T03:40:29.906Z| vmx| I125: GuestRpc: Got error for channel 1 connection 21: Remote disconnected
2017-08-03T03:40:30.921Z| vmx| I125: GuestRpc: Got error for channel 1 connection 22: Remote disconnected
2017-08-03T03:40:31.937Z| vmx| I125: GuestRpc: Got error for channel 1 connection 23: Remote disconnected
2017-08-03T03:40:32.937Z| vmx| I125: GuestRpc: Got error for channel 1 connection 24: Remote disconnected
Code:
2017-08-03T03:43:32.166Z| vcpu-3| I125: VMXNET3 user: Ethernet0 Driver Info: version = 4062452 gosBits = 2 gosType = 2, gosVer = 25344, gosMisc = 212
2017-08-03T03:43:36.315Z| vmx| I125: GuestRpc: Got error for channel 1 connection 205: Remote disconnected
2017-08-03T03:45:17.925Z| vmx| I125: GuestRpc: Got error for channel 1 connection 305: Remote disconnected
2017-08-03T03:47:01.942Z| vmx| I125: GuestRpc: Got error for channel 1 connection 405: Remote disconnected
 

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
Code:
 mks| W115: low memory mode
2017-08-03T04:28:50.622Z| mks| W115: low memory mode
2017-08-03T04:28:50.623Z| mks| W115: low memory mode
2017-08-03T04:28:59.952Z| mks| W115: low memory mode
2017-08-03T04:30:29.437Z| vmx| I125: GuestRpc: Got error for channel 2 connection 3006: Remote disconnected
2017-08-03T04:30:49.196Z| svga| I125: MKSScreenShotMgr: Taking a screenshot
2017-08-03T04:31:03.209Z| vcpu-2| I125: GuestRpc: Reinitializing Channel 1(toolbox-dnd)
2017-08-03T04:31:03.844Z| vmx| I125: TOOLS installed legacy version 10272, available legacy version 10272
2017-08-03T04:31:03.844Z| vmx| I125: TOOLS manifest update status is 3
2017-08-03T04:31:03.844Z| vmx| I125: TOOLS can be autoupgraded.
2017-08-03T04:31:03.846Z| vmx| I125: TOOLS autoupgrade protocol version 0
 

realtomatoes

Active Member
Oct 3, 2016
245
31
28
41
did you backup before upgrading?

is the vm's OS on a vmdk or on one of the passthrough drives? if in a vmdk, i'd start by removing the passthrough hardware and see if that boots up.
 

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
I do have a backup, but i already moved data off the passthrough drives to new vmfs drives. I also removed all pass-through devices.
Seems like something is taking the memory and then causing the system to crash.
 

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
windows server boots up fine. im trying to run a backup and after a while the vm crashes. I uninstalled vmware tools and trying it again. ill let it run over night to see if it crashes again. now its time for bed.
 

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
So i left the server running last night with backups going and no vmware tools installed. So far none of the above messages were in the vmware.log and it hasnt crashed.

I'm hoping to let the backups finish then ill try to reinstall vmware tools to see if it happens again.
 

marcoi

Well-Known Member
Apr 6, 2013
1,406
226
63
Gotha Florida
I restored from an old backup that didnt have the hardware version upgraded and so far the vm hasnt crashed. Not sure why upgrading the hardware may have caused issues.