Warning - Disk 8 has been surprise removed

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Fritz

Well-Known Member
Apr 6, 2015
3,382
1,385
113
70
Getting this in the system event log. There is no Disk 8 nor has there ever been a disk 8 so WTF is Windows Server 2012 R2 Standard talking about?
 

dicecca112

Active Member
Feb 10, 2016
302
36
28
39
Can you post the entire log?

Also can you post the output of diskpart command list disks
 

DavidRa

Infrastructure Architect
Aug 3, 2015
330
153
43
Central Coast of NSW
www.pdconsec.net
Have you had a USB device plugged in? IPMI USB perhaps?

I do like VMware saying "We did a thing and it caused a message in Windows, but it's TOTALLY Microsoft's fault". Hint guys: If you do a thing, and a message occurs in the logs, and it happens every time, IT'S YOUR FAULT.
 

Fritz

Well-Known Member
Apr 6, 2015
3,382
1,385
113
70
Not running VMware on this box, just straight Windows Server 2012 R2 Standard. The only software it runs is Blue Iris. No flash drives or any other removable devices involved.

A closer look reveals that this happens in clusters -

1. The Portable Device Enumerator Service service entered the running state.
2. Disk 8 has been surprise removed.
3. Disk 8 has been surprise removed.
4. Disk 8 has been surprise removed.
5. The Portable Device Enumerator Service service entered the stopped state.
6. The Portable Device Enumerator Service service entered the running state.
7. Disk 8 has been surprise removed.

1, 5 and 6 are information. 2, 3, 4 and 7 are warnings. This is too consistent to be a random problem, something is causing it, but what?? :confused:

I'm getting ready to move Blue Iris to another box because this 1U chassis is proving to be impossible to keep cool in the environment it's in. CPU's stay at "Medium" and occasionally hit "High". The fans speed up and slow down erratically.
 

Fritz

Well-Known Member
Apr 6, 2015
3,382
1,385
113
70
OS = Windows Server 2012 R2 Standard
MB = Supermicro X8DTL-3F
Mem = 48GB DDR3 ECC RDIMM (8 x 6GB)
Boot Drive = MK0400GCTZA (HP Intel DC S3700 400GB)
Storage = 8 x HGST 450GB SAS (HUC106045CSS601)

No additional tasks scheduled.
 

DavidRa

Infrastructure Architect
Aug 3, 2015
330
153
43
Central Coast of NSW
www.pdconsec.net
OK I interpret the event log as having a backup or shadow copy at the times just before/after the event. Example:

Code:
Information    9/13/2016 11:00:00 PM    Virtual Disk Service    3    None    Service started.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Block Level Backup Engine Service service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Virtual Disk service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Volume Shadow Copy service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Microsoft Software Shadow Copy Provider service entered the running state.
Information    9/13/2016 11:00:18 PM    Microsoft-Windows-Ntfs    98    None    Volume System Reserved (\Device\HarddiskVolumeShadowCopy43) is healthy.  No action is needed.
Information    9/13/2016 11:00:18 PM    Microsoft-Windows-Ntfs    98    None    Volume ?? (\Device\HarddiskVolumeShadowCopy44) is healthy.  No action is needed.
Information    9/13/2016 11:01:12 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the running state.
Warning    9/13/2016 11:01:12 PM    disk    157    None    Disk 8 has been surprise removed.
Warning    9/13/2016 11:01:16 PM    disk    157    None    Disk 8 has been surprise removed.
Warning    9/13/2016 11:01:17 PM    disk    157    None    Disk 8 has been surprise removed.
Information    9/13/2016 11:03:17 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the stopped state.
Information    9/13/2016 11:04:29 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the running state.
Warning    9/13/2016 11:04:30 PM    disk    157    None    Disk 8 has been surprise removed.
Information    9/13/2016 11:04:33 PM    Service Control Manager    7036    None    The Device Setup Manager service entered the running state.
Information    9/13/2016 11:06:30 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the stopped state.
Information    9/13/2016 11:06:35 PM    Service Control Manager    7036    None    The Device Setup Manager service entered the stopped state.
Information    9/13/2016 11:07:38 PM    Service Control Manager    7036    None    The Volume Shadow Copy service entered the stopped state.
Information    9/13/2016 11:10:00 PM    Virtual Disk Service    4    None    Service stopped.
Information    9/13/2016 11:10:00 PM    Service Control Manager    7036    None    The Block Level Backup Engine Service service entered the stopped state.
Information    9/13/2016 11:10:00 PM    Service Control Manager    7036    None    The Virtual Disk service entered the stopped state.
Information    9/13/2016 11:10:38 PM    Service Control Manager    7036    None    The Microsoft Software Shadow Copy Provider service entered the stopped state.
I'd suggest checking for shadow copies on the data and system disks; Windows Backup configuration on this server or another one (perhaps something taking a remote backup at 11pm?) - it's ALWAYS 11:00pm this starts.
 
  • Like
Reactions: T_Minus and Fritz

Fritz

Well-Known Member
Apr 6, 2015
3,382
1,385
113
70
OK I interpret the event log as having a backup or shadow copy at the times just before/after the event. Example:

Code:
Information    9/13/2016 11:00:00 PM    Virtual Disk Service    3    None    Service started.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Block Level Backup Engine Service service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Virtual Disk service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Volume Shadow Copy service entered the running state.
Information    9/13/2016 11:00:00 PM    Service Control Manager    7036    None    The Microsoft Software Shadow Copy Provider service entered the running state.
Information    9/13/2016 11:00:18 PM    Microsoft-Windows-Ntfs    98    None    Volume System Reserved (\Device\HarddiskVolumeShadowCopy43) is healthy.  No action is needed.
Information    9/13/2016 11:00:18 PM    Microsoft-Windows-Ntfs    98    None    Volume ?? (\Device\HarddiskVolumeShadowCopy44) is healthy.  No action is needed.
Information    9/13/2016 11:01:12 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the running state.
Warning    9/13/2016 11:01:12 PM    disk    157    None    Disk 8 has been surprise removed.
Warning    9/13/2016 11:01:16 PM    disk    157    None    Disk 8 has been surprise removed.
Warning    9/13/2016 11:01:17 PM    disk    157    None    Disk 8 has been surprise removed.
Information    9/13/2016 11:03:17 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the stopped state.
Information    9/13/2016 11:04:29 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the running state.
Warning    9/13/2016 11:04:30 PM    disk    157    None    Disk 8 has been surprise removed.
Information    9/13/2016 11:04:33 PM    Service Control Manager    7036    None    The Device Setup Manager service entered the running state.
Information    9/13/2016 11:06:30 PM    Service Control Manager    7036    None    The Portable Device Enumerator Service service entered the stopped state.
Information    9/13/2016 11:06:35 PM    Service Control Manager    7036    None    The Device Setup Manager service entered the stopped state.
Information    9/13/2016 11:07:38 PM    Service Control Manager    7036    None    The Volume Shadow Copy service entered the stopped state.
Information    9/13/2016 11:10:00 PM    Virtual Disk Service    4    None    Service stopped.
Information    9/13/2016 11:10:00 PM    Service Control Manager    7036    None    The Block Level Backup Engine Service service entered the stopped state.
Information    9/13/2016 11:10:00 PM    Service Control Manager    7036    None    The Virtual Disk service entered the stopped state.
Information    9/13/2016 11:10:38 PM    Service Control Manager    7036    None    The Microsoft Software Shadow Copy Provider service entered the stopped state.
I'd suggest checking for shadow copies on the data and system disks; Windows Backup configuration on this server or another one (perhaps something taking a remote backup at 11pm?) - it's ALWAYS 11:00pm this starts.
Thank you sir. Excellent detective work. when I set this server up, several months ago, I also set up Server Backup and promptly forgot about it. Backups are occurring successfully so still no idea why I'm seeing this message but at least I now know where it's coming from.
 

Jeff Robertson

Active Member
Oct 18, 2016
429
115
43
Chico, CA
I see this all the time on my hyper-v cluster and DavidRa is correct, it has to do with VSS. This happens on each of my VMs when a backup is triggered, the OS seems surprised the disk has "changed."
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Isn't just hyper-V that does this, there's a load of backup sproducts on vmware that cause this also. As others have said, it's due to VSS snapshots being deleted ("surprise removed") when the rest of the OS, for some reason, isn't expecting it to.

Bonus points for anyone who's ever got an "NTFS delayed write failed" when VSS tried to quiesce the non-existent floppy drive.
 
  • Like
Reactions: Fritz