NAS02 Server specs:
OS: Windows Server 2016
Chassis: Supermicro SC846E16-R1200B
PSU: Redundant 920W PWS-920P-1R 80PLUS Platinum
Mobo: ASRock E3C224 ATX
CPU: Intel E3-1241v3 3.5GHz
RAM: 32 GB Kingston 4x8 GB DDR3 ECC Unbuffered KVR16E11/8
NIC: Mellanox ConnectX-2 (Directly connected to my primary NAS, NAS01)
RAID: IBM M5015 w/ BBU
SSD: 120 GB Samsung 840 Evo - OS
HDD: 14x 4 TB HGST NAS (Raid 6) - Primary backup pool
This server is my backup server of a similarly spec'd NAS (NAS01) that houses all of my... stuff. NAS02 stays powered off throughout the day, but at 01:00 every night, a scheduled task on NAS01 runs a script to power on NAS02 via IPMI (if it's not already on), waits for NAS02 to come online, and then executes a GoodSync job to copy any new or modified data to NAS02. After the backup job completes, the script on NAS01 will send a shutdown command to NAS02, but only if it was powered off already before the script executed. I'm not the best at scripting, but it works well...
HOWEVER... Every night, after the backup job finishes, and NAS02 has shutdown, anywhere between 20 minutes and a couple hours later, NAS02 will power back on by itself. I've checked the Windows event viewer, and it shows nothing other than a standard power-on, as if someone powered on the server manually. I checked the logs in my motherboard's IPMI, and also see a normal power-on. The logs don't show if or what user powered on the server (below). Even when I power on the server in the IPMI console manually, using my admin account, it doesn't log and say that the admin sent the power-on command.
09/15/2017 02:47:22 Unknown OS Boot C: Boot Completed - Asserted
I'm at a loss. I have no idea why I'm getting these phantom boots on this server. I went ahead and set the 'after power failure' setting in my BIOS to 'Off', as it was 'last state' before. This didn't fix it either. I have two Cyberpower UPS' for three servers (my host running the center install), and looked in the center's and client's logs, but also don't see any indication that the power-on came from the UPS software.
What do I try next?
My scripts and last night's log is below.
7xNAS02-GSync-Script.bat (pardon the mess)
7xNAS02-Power-On.ps1
Log output from script:
EDIT: After posting this, I found that my BIOS has a Boot from Onboard LAN feature that was disabled, so it wasn't that. However, I did see that 'Allow this device to wake the computer' was checked in my NIC's power management settings. I went ahead and disabled that.
OS: Windows Server 2016
Chassis: Supermicro SC846E16-R1200B
PSU: Redundant 920W PWS-920P-1R 80PLUS Platinum
Mobo: ASRock E3C224 ATX
CPU: Intel E3-1241v3 3.5GHz
RAM: 32 GB Kingston 4x8 GB DDR3 ECC Unbuffered KVR16E11/8
NIC: Mellanox ConnectX-2 (Directly connected to my primary NAS, NAS01)
RAID: IBM M5015 w/ BBU
SSD: 120 GB Samsung 840 Evo - OS
HDD: 14x 4 TB HGST NAS (Raid 6) - Primary backup pool
This server is my backup server of a similarly spec'd NAS (NAS01) that houses all of my... stuff. NAS02 stays powered off throughout the day, but at 01:00 every night, a scheduled task on NAS01 runs a script to power on NAS02 via IPMI (if it's not already on), waits for NAS02 to come online, and then executes a GoodSync job to copy any new or modified data to NAS02. After the backup job completes, the script on NAS01 will send a shutdown command to NAS02, but only if it was powered off already before the script executed. I'm not the best at scripting, but it works well...
HOWEVER... Every night, after the backup job finishes, and NAS02 has shutdown, anywhere between 20 minutes and a couple hours later, NAS02 will power back on by itself. I've checked the Windows event viewer, and it shows nothing other than a standard power-on, as if someone powered on the server manually. I checked the logs in my motherboard's IPMI, and also see a normal power-on. The logs don't show if or what user powered on the server (below). Even when I power on the server in the IPMI console manually, using my admin account, it doesn't log and say that the admin sent the power-on command.
09/15/2017 02:47:22 Unknown OS Boot C: Boot Completed - Asserted
I'm at a loss. I have no idea why I'm getting these phantom boots on this server. I went ahead and set the 'after power failure' setting in my BIOS to 'Off', as it was 'last state' before. This didn't fix it either. I have two Cyberpower UPS' for three servers (my host running the center install), and looked in the center's and client's logs, but also don't see any indication that the power-on came from the UPS software.
What do I try next?
My scripts and last night's log is below.
7xNAS02-GSync-Script.bat (pardon the mess)
Code:
@echo off
REM Change CMD window color and size.
color 37
mode 180,50
REM ### Set variables
SET ipaddress1=11.11.11.20 REM ### Use the 10GbE NIC to ping 7xNAS02
SET ipaddress2=10.0.3.1
SET LOGFILE=C:\!SCRIPTS\7xNAS02-GSync-Log.txt
set DATE=%DATE:~4,10%
set dateStr=%DATE:~6,4%%DATE:~3,2%%DATE:~0,2%
:START
REM ### Checks if 7xNAS02 is online. If not, power it on.
ECHO ################################################################################################################################ >> %LOGFILE%
ECHO %DATE% %time% EXECUTING GSYNC SCRIPT >> %LOGFILE%
ECHO ################################################################################################################################ >> %LOGFILE%
ECHO.
ECHO This is an automated script to backup the storage array on 7xNAS01 onto 7xNAS02.
ECHO If 7xNAS02 isn't powered on, the script will boot 7xNAS02, wait for it to come online,
ECHO and then execute the "STORAGE" backup job in GoodSync.
ECHO.
ECHO %DATE% %time% Checking power status of 7xNAS02... >> %LOGFILE%
ECHO Checking power status of 7xNAS02...
ECHO.
ping -n 1 %ipaddress1% | findstr TTL && goto ALREADY_ON
ping -n 1 %ipaddress1% | findstr TTL || goto POWER_ON
:ALREADY_ON
REM Sets variable to dictate that the server was already on before executing the script.
ECHO.
ECHO %DATE% %time% 7xNAS02 is already powered on and available. >> %LOGFILE%
ECHO 7xNAS02 is already powered on and available.
ECHO.
SET previously_off=1
goto EXECUTE_BACKUP
:POWER_ON
REM ### Powers on 7xNAS02
ECHO %DATE% %time% 7xNAS02 is offline. Powering on now. >> %LOGFILE%
ECHO 7xNAS02 is offline. Powering on now.
ECHO.
SET previously_off=0
PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& 'C:\!SCRIPTS\7xNAS02-Power-On.ps1'" >> %LOGFILE%
ECHO %DATE% %time% Pinging 7xNAS02 until it's available... >> %LOGFILE%
ECHO Pinging 7xNAS02 until it's available...
ECHO.
goto CHECK_IF_READY
:CHECK_IF_READY
REM ### Continually pings 7xNAS02 until it is online.
ping -n 1 %ipaddress1% | findstr TTL || goto CHECK_IF_READY
ping -n 1 %ipaddress1% | findstr TTL && goto EXECUTE_BACKUP
:EXECUTE_BACKUP
REM ### Executes GoodSync backup job, "STORAGE."
ECHO %DATE% %time% Executing "STORAGE" backup job. >> %LOGFILE%
ECHO Executing "STORAGE" backup job.
ECHO.
c:\"Program Files"\"Siber Systems"\Goodsync\gsync sync "STORAGE" >> %LOGFILE%
if %previously_off% == 0 (
ECHO.
ECHO %DATE% %time% Sync job done. Shutting down 7xNAS02... >> %LOGFILE%
ECHO Sync job done. Shutting down 7xNAS02...
shutdown /m \\7xNAS02 /s /f /d p:0:0 /c "Executed by 7xNAS02-GSync-Script on 7xNAS01."
goto SHUTDOWNCHECK
)
if %previously_off% == 1 (
ECHO.
ECHO %DATE% %time% Sync job done. 7xNAS02 will stay powered on. >> %LOGFILE%
ECHO Sync job done. 7xNAS02 will stay powered on.
ECHO.
)
goto END
:SHUTDOWNCHECK
ping -n 5 %ipaddress1% >nul
if ERRORLEVEL 1 (
REM ### 7xNAS02 has powered off.
ECHO.
ECHO %DATE% %time% 7xNAS02 has powered off. >> %LOGFILE%
ECHO 7xNAS02 has powered off.
goto END
)
REM ### 7xNAS02 is still on. Looping to check if the server has shut down.
ECHO ...
goto SHUTDOWNCHECK
:END
REM pause
ECHO %DATE% %time% END OF SCRIPT >> %LOGFILE%
exit
Code:
$username = "username"
$password = "password"
$secstr = New-Object -TypeName System.Security.SecureString
$password.ToCharArray() | ForEach-Object {$secstr.AppendChar($_)}
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $secstr
$Ip = "10.0.3.99"
Get-PcsvDevice -TargetAddress $Ip -ManagementProtocol IPMI -Credential $Cred | Start-PcsvDevice
Code:
################################################################
09/15/2017 1:00:00.86 EXECUTING GSYNC SCRIPT
################################################################
09/15/2017 1:00:00.87 Checking power status of 7xNAS02...
09/15/2017 1:00:08.82 7xNAS02 is offline. Powering on now.
09/15/2017 1:00:12.20 Pinging 7xNAS02 until it's available...
09/15/2017 1:02:20.92 Executing "STORAGE" backup job.
*REDACTED*
09/15/2017 2:24:13.30 Sync job done. Shutting down 7xNAS02...
09/15/2017 2:25:33.82 7xNAS02 has powered off.
09/15/2017 2:25:33.82 END OF SCRIPT
Last edited: